-- Logs begin at Mon 2022-08-01 06:48:08 UTC, end at Mon 2022-08-01 07:55:14 UTC. -- Aug 01 07:22:58.733870 dv-cinder-733062-31-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 07:22:58.741856 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 07:22:58 2022] *** Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: nodename: dv-cinder-733062-31-iscsi Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: machine: x86_64 Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: clock source: unix Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: pcre jit disabled Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: detected number of CPU cores: 4 Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: current working directory: / Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: detected binary path: /usr/bin/uwsgi-core Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: your processes number limit is 47680 Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: your memory page size is 4096 bytes Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: detected max file descriptor number: 2048 Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: lock engine: pthread robust mutexes Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: thunder lock: enabled Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 07:22:58.747088 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 07:22:58.785963 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: Python main interpreter initialized at 0x55d39d9689c0 Aug 01 07:22:58.785963 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: python threads support enabled Aug 01 07:22:58.785963 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: your server socket listen backlog is limited to 100 connections Aug 01 07:22:58.785963 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: your mercy for graceful operations on workers is 90 seconds Aug 01 07:22:58.786358 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 07:22:58.786429 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: *** Operational MODE: preforking *** Aug 01 07:22:58.786658 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: *** uWSGI is running in multiple interpreter mode *** Aug 01 07:22:58.786658 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: spawned uWSGI master process (pid: 80145) Aug 01 07:22:58.786859 dv-cinder-733062-31-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 07:22:58.788278 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: spawned uWSGI worker 1 (pid: 80159, cores: 1) Aug 01 07:22:58.788278 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: spawned uWSGI worker 2 (pid: 80160, cores: 1) Aug 01 07:22:58.788278 dv-cinder-733062-31-iscsi devstack@keystone.service[80145]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 07:23:04.389802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 07:23:04.389802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: from cryptography.utils import int_from_bytes Aug 01 07:23:04.391205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 07:23:04.391205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: from cryptography.utils import int_from_bytes Aug 01 07:23:04.440257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 07:23:04.440447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 07:23:04.440638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] command line args: [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 07:23:04.440909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 07:23:04.441087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ================================================================================ {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 07:23:04.441345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] admin_token = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.441594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] config_dir = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.441783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.441967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] config_source = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.442148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] debug = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.442346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.442840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.442925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.443156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.443406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.443638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.443817 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_config_append = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.443995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.444224 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_dir = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.444339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_file = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.444578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.444825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.445072 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.445318 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.445498 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.445671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.445916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.446166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.446401 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.446634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.446867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.447120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.447369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.447455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 07:23:04.447455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: from cryptography.utils import int_from_bytes Aug 01 07:23:04.447936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.448008 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 07:23:04.448008 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: from cryptography.utils import int_from_bytes Aug 01 07:23:04.448124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.448365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.448546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] publish_errors = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.448779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.449016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.449195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.449444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.449678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.449912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] standard_threads = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.450145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.450387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.450564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.450740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] use_journal = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.450915 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] use_json = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.451112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] use_stderr = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.451290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] use_syslog = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.451521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.451779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.452019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.452280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.452521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.452773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.453532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.453799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.454159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.454416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.454669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.454923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.455186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.455441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.455681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.455932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.456183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.456420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.456678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.456936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.457184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.457422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.457678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.457982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.458243 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.458492 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.458727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.458961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.459217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.459460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.459710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.459959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.460192 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.460425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.460674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.460924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.461164 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.461413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.461647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.461883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.462119 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.462353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.462586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.462818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.463049 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.463308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.463559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.463791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.464028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.464258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.464495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.464733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.464983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.465215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.465452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.465689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.465925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.466160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.466397 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.466628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.466879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.467136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.467375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.467608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.467867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.468103 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.468334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.468634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.469439 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.469637 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.469986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.470247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.470495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.470734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.470973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.471234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.471487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.471728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.471969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.472213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.472452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.472690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.472927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.473151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.473388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.473650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.474043 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.474185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.474348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.474590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.474827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.475094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.475319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.475552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.475784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.476016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.476253 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.476484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.476725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.476962 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.477210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.477445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.477700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.477961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.478203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.478436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.external = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.478666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.478966 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.479221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.479460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.password = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.479691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] auth.token = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.479931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.480168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.480416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.480650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.480899 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.481147 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.481379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.481618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.481868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.482118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.482365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.482602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.482848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.483107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.483381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.483636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.483875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.484158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.484454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.484702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.484977 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.485222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.485495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.485735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.485975 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.486225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.486474 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.486706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.486953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.487214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.487473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.487711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.487969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.488208 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.488464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.488701 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.488950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.489199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.489441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.489680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.489941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.490148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.490387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.490637 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.490908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.491146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.491381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.491613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.491856 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.492104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.492353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.492609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.492856 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.493126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.493365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.493603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.493834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.494085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.494300 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.494535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.494776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.495014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.495289 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.495520 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.495768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.496015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.496205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 07:23:04.496353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.496489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 07:23:04.496618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.496746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] command line args: [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 07:23:04.496871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.497003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 07:23:04.497131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.497265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ================================================================================ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 07:23:04.497393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.497519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] admin_token = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.497645 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.497771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_dir = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.497904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.498042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.498186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.498314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_source = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.498314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] debug = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.498561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.498696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.498854 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.498982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.499144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.499279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.499407 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.499542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.499670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.499812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.499950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.500078 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.500204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.500340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_config_append = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.500340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.500580 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.500706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_dir = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.500832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.500959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.501093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.501093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.501342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.501478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.501605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.501731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.501865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.501994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.502121 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.502249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.502375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.502511 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.502644 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.502773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.502899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.503027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.503189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.503189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.503447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.503574 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.503706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.503706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.503959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.503959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.504250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.504250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.504494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.504621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.504747 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.504883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.505018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.505149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.505149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.505395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.505523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.505658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] publish_errors = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.505658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.505899 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.506025 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.506151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.506287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.506415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.506550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.506677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.506809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.506943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.507101 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] standard_threads = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.507317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.507449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.507576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.507703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.507703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.507986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.508135 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_journal = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.508276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.508406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_json = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.508406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_stderr = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.508658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.508786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_syslog = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.508912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.509051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 07:23:04.509180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.509308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.509435 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.509562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.509688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.509834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.509967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.510095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.510222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.510386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.510516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.510653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.510781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.510908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.511034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.511221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.511360 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.511504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.511634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.511761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.511888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.512021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.512160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.512296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.512425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.512552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.512680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.513064 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.513205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.513343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.513472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.513600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.513727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.513854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.513982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.514117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.514263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.514399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.514527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.514527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] role.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.514769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.514769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.515019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] role.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.515019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.515288 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.515945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.516125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.516269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.516401 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.516540 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.516670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.516798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.516934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.517064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.517198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.517198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.517441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.517568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.517705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.517705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.517949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.518076 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.518204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.518342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.518479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.518479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.518735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.518872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.519000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.519000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.519285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.519414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.519542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.519669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.519807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.519935 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.520155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.520287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.520415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.520542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.520669 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.520796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.520934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.521062 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.521190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.521317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.521445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.521571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.521715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.521855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.521990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.522118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.522245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.522372 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.522507 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.522645 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.522773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.522900 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.523034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.523188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.523330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.523460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.523587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.523724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.523853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.523979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.524115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.524283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.524419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.524548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.524676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.524802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.524938 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.525068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.525195 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.525322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.525458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.525595 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.525723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.525850 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.525976 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.526104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.526231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.526366 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.526501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.526630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.526757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.526892 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.527020 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.527176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.527327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.527457 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.527584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.527711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.527838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.527965 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.528100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.528229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.528356 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.528482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.528615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.528743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.528878 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.529022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.529150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.529278 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.connection = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.529413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.529551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.529686 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.529830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.529960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.530086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.530223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.530359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.530492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.530628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.530758 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.530896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.531672 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.531812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532418 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.532924 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533075 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533511 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 07:23:04.533794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.533794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.534337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.534337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.534337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.534337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.534337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.external = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.534337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.534337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.534337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.534885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.password = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.534885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.token = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.535020 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.535290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.535544 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.535783 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.536038 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.536292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.536529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.536773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.537031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.537287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.537540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.537781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.538029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.538270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.538541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.538801 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.539042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.539322 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.539610 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.539861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.540129 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.540375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.540637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.540879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.541119 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.541371 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.541622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.541858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.542108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.542348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.542607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.542848 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.543208 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.543454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.543706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.543948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.544201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.544453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.544693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.544934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.545186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.545424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.545665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.545917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.546171 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.546410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.546844 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.547345 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.547630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.547889 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.548145 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.548408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.548660 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.548935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.549180 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.549421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.549657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.549898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.550134 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.550374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.550616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.550859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.551134 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.551371 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.551622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.551874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.552114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.552377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.552630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.552866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.553107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.553341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.553583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.553825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.554070 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.554310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.554551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.554804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.555055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.555312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.555550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.555805 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.556056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.556296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.556536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.556775 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.557013 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.557255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.557518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.557755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.558007 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.558258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.558507 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.558747 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.558981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.559246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.559391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.559532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.559670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.559805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.559964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.559964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.560232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.560406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.560655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.560904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.561152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.561385 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.561636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.561876 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.562115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.562968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.563433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.563728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.563985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.564228 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.564490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.564726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.564978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.565220 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.565462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.565819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.566073 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.566331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.566569 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.566810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.567055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.567330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.567590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.567831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.568086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.568338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.568580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.568721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.568860 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.569061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.569304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.569545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.569797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.570031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.570283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.570534 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.570785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.571035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.571309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.571559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.571808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.572058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.572294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.572546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.572797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.573048 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.573301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.573723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.573964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.574218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.574471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.574724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.575138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.575295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.575469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.575725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.575970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.576206 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.576442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.576676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.576928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.577185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.577441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.577683 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.578630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.579100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.579398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.579646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.579906 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.580149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.580413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.580670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.580915 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.581167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.581417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.581670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.581912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.582156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.582395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.582645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.582897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.583153 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.583402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.583707 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.583971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.584221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.584472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.584712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.584958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.585197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.585438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.585677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.585917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.586164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.586403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.586649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.586888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.587165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.587407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.587654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.587888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.588137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.588377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 07:23:04.588549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 07:23:04.598204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.598439 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.613314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.613550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.613782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.622136 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.643518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.643701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.650967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.681138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.701224 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.701459 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.719572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.719941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.767326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.767562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.767790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.768015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.768098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.768239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.768324 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.768458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.768541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.768713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.768948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.769156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.791459 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.791702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.791924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.792084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.792189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.792325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.792408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.792560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.792642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.792773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.792856 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.792993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.793211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.793428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.803709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.803809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.817057 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.817290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.817502 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.817586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.817729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.817813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.817950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.818031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.818166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.818262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.818383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.818464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80159) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.818604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.818821 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 07:23:04.835042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.835282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 07:23:04.835614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.835813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 07:23:04.835922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.836078 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.836176 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 07:23:04.836334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.836457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.836587 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.836668 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 07:23:04.836841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.836929 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.837170 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.837429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.837685 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.847250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.847527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.847793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.848073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.848203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.848481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.848749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.849030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.854863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.855914 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.859705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.859982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.860262 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.860544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.860824 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.861079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.861160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.861344 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.861427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.861612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.861693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.861881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.861969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.862152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.862232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.862420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.862498 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.862693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.862970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.863272 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.863547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.863816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.885678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.887177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.887348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.887443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.888980 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.889258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.892475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.892752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.893038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.894340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.894581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.894864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.899631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.899913 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.901387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.901673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.903981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.904268 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.904558 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.904837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.905108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.905387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.905655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.905932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.907651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.907942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.908235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.908527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.908806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.909090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.909370 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.909645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.922183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.922469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.925805 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.927524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.927654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.929462 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.929754 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.929987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.933155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.933451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.934075 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.937275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.938091 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.940541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.940841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.941126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.941294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.941411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.941695 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.941977 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.944527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.944829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.945113 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.945412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.945700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.945978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.958381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.958667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.958934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.959224 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.962761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.963051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.963338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.964069 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.966018 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.967568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.967845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.971189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.971331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.971468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.971766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.972056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.972344 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.972838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.973122 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.975934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.976216 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.976507 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.976799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.977101 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.983570 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.983848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.984166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.984450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.984735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.984997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.985267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.988858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:04.989141 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.989426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.989711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.989994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.990272 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:04.990549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.000602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:05.003528 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:05.006413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.006512 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:05.008932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.009398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:05.010546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:05.010829 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.011130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.012355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.015183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.017505 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:05.017799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.018081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.018247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.019849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:05.022709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.023000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.023315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.023601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.024546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.026143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:05.029044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.029347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.029639 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.029912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.032059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:05.032416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.032708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.038850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:05.039160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.039449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.039534 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80159) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:05.039742 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.040021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.040296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.040569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.040859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.041141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.041416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.041783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.042068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.042346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.042617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.042890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.043179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80159) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.046198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 07:23:05.046477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.047031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.047386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.047678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.047953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.048227 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.048497 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.048879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.049164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.049449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.049721 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.049989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.050251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 07:23:05.076197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d39d9689c0 pid: 80159 (default app) Aug 01 07:23:05.080001 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e80d9d-906c-4642-9e1e-2bad8ce2bf54 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:05.080289 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e80d9d-906c-4642-9e1e-2bad8ce2bf54 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:05.080565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e80d9d-906c-4642-9e1e-2bad8ce2bf54 None None] PATH_INFO: `/v3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:05.084427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d39d9689c0 pid: 80160 (default app) Aug 01 07:23:05.084812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Mon Aug 1 07:23:05 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:12.660762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0c903-db4d-4673-b8a8-d7d0f1885218 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:12.661328 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0c903-db4d-4673-b8a8-d7d0f1885218 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:12.661328 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0c903-db4d-4673-b8a8-d7d0f1885218 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:12.665746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:12.670869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0898bdf-e2db-4091-9bf0-c7b985b1953c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:12.671960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0898bdf-e2db-4091-9bf0-c7b985b1953c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:12.672231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0898bdf-e2db-4091-9bf0-c7b985b1953c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:12.742215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: INFO dbcounter [None req-c0898bdf-e2db-4091-9bf0-c7b985b1953c None None] Registered counter for database keystone Aug 01 07:23:12.761661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG oslo_db.sqlalchemy.engines [None req-c0898bdf-e2db-4091-9bf0-c7b985b1953c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80159) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 07:23:12.768133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writer thread running {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 07:23:12.945237 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG passlib.handlers.bcrypt [None req-c0898bdf-e2db-4091-9bf0-c7b985b1953c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80159) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 07:23:12.945672 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG passlib.handlers.bcrypt [None req-c0898bdf-e2db-4091-9bf0-c7b985b1953c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80159) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 07:23:13.000748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c0898bdf-e2db-4091-9bf0-c7b985b1953c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:13.002367 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c0898bdf-e2db-4091-9bf0-c7b985b1953c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:13.132850 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 01 07:23:13.256612 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0c903-db4d-4673-b8a8-d7d0f1885218 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:13.324975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: INFO dbcounter [None req-f3b0c903-db4d-4673-b8a8-d7d0f1885218 None None] Registered counter for database keystone Aug 01 07:23:13.346684 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG oslo_db.sqlalchemy.engines [None req-f3b0c903-db4d-4673-b8a8-d7d0f1885218 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80160) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 07:23:13.352855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writer thread running {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 07:23:13.472713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0c903-db4d-4673-b8a8-d7d0f1885218 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:13.476988 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d509c-2004-4363-8652-21386fb828ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:13.477625 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d509c-2004-4363-8652-21386fb828ea admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:13.477916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d509c-2004-4363-8652-21386fb828ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:13.478198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d509c-2004-4363-8652-21386fb828ea admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:13.486604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d509c-2004-4363-8652-21386fb828ea admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:13.539495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f16d509c-2004-4363-8652-21386fb828ea admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 07:23:13.539788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d509c-2004-4363-8652-21386fb828ea admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:13.549428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Mon Aug 1 07:23:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:23:13.555334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0898bdf-e2db-4091-9bf0-c7b985b1953c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:13.570199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0898bdf-e2db-4091-9bf0-c7b985b1953c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:13.574561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2cf02-00c9-48ae-911f-124e26414bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:13.575703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2cf02-00c9-48ae-911f-124e26414bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:13.575987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2cf02-00c9-48ae-911f-124e26414bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:13.576271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2cf02-00c9-48ae-911f-124e26414bd5 admin admin] PATH_INFO: `/v3/projects/a9fa482966924d01a3dc7809fa925311` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:13.578389 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2cf02-00c9-48ae-911f-124e26414bd5 admin admin] RBAC: Authorizing `identity:get_project(project_id=a9fa482966924d01a3dc7809fa925311)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:13.628706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: WARNING keystone.common.rbac_enforcer.enforcer [None req-afd2cf02-00c9-48ae-911f-124e26414bd5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 07:23:13.629042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2cf02-00c9-48ae-911f-124e26414bd5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:13.633706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Mon Aug 1 07:23:13 2022] GET /identity/v3/projects/a9fa482966924d01a3dc7809fa925311 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:15.619115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffc54f-5350-41ee-a7df-3f05f49a4ace None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:15.619657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffc54f-5350-41ee-a7df-3f05f49a4ace None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:15.619657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffc54f-5350-41ee-a7df-3f05f49a4ace None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:15.623787 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:15.629805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0848d-691a-40e0-86ca-5effe185619d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:15.630226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0848d-691a-40e0-86ca-5effe185619d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:15.630372 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0848d-691a-40e0-86ca-5effe185619d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:15.650302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-95f0848d-691a-40e0-86ca-5effe185619d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:15.651859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-95f0848d-691a-40e0-86ca-5effe185619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:15.659799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 07:23:15.681562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffc54f-5350-41ee-a7df-3f05f49a4ace None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:15.693989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffc54f-5350-41ee-a7df-3f05f49a4ace None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:15.698215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7fe69-fb2d-492f-8d1d-9f6762f3c90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:15.698814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7fe69-fb2d-492f-8d1d-9f6762f3c90c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:15.699131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7fe69-fb2d-492f-8d1d-9f6762f3c90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:15.699421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7fe69-fb2d-492f-8d1d-9f6762f3c90c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:15.706951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7fe69-fb2d-492f-8d1d-9f6762f3c90c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:15.708862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7fe69-fb2d-492f-8d1d-9f6762f3c90c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:15.717416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Mon Aug 1 07:23:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:23:15.722685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0848d-691a-40e0-86ca-5effe185619d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:15.731674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0848d-691a-40e0-86ca-5effe185619d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:15.736710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518234f-25cc-4655-b342-514fa3a24326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:15.737337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518234f-25cc-4655-b342-514fa3a24326 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:15.737625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518234f-25cc-4655-b342-514fa3a24326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:15.737906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518234f-25cc-4655-b342-514fa3a24326 admin admin] PATH_INFO: `/v3/users/aae3d76a87c24d97b28248f570423938` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:15.739350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518234f-25cc-4655-b342-514fa3a24326 admin admin] RBAC: Authorizing `identity:get_user(user_id=aae3d76a87c24d97b28248f570423938)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:15.741177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518234f-25cc-4655-b342-514fa3a24326 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:15.742719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Mon Aug 1 07:23:15 2022] GET /identity/v3/users/aae3d76a87c24d97b28248f570423938 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:17.779086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98e37a-8f36-496a-bf44-8d60833ba94c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:17.779630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98e37a-8f36-496a-bf44-8d60833ba94c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:17.779630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98e37a-8f36-496a-bf44-8d60833ba94c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:17.781108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:17.786279 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb31857-c548-4442-a869-f53c9b7fae2e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:17.786671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb31857-c548-4442-a869-f53c9b7fae2e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:17.786841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb31857-c548-4442-a869-f53c9b7fae2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:17.806019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6cb31857-c548-4442-a869-f53c9b7fae2e None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:17.807267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6cb31857-c548-4442-a869-f53c9b7fae2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:17.828861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89d100-590f-4f14-9cd7-ecd654a7ff1e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:17.829149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89d100-590f-4f14-9cd7-ecd654a7ff1e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:17.829433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89d100-590f-4f14-9cd7-ecd654a7ff1e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:17.830772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:23:17.835948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91750879-6b28-4d21-9f50-9032c285ffbe None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:17.836236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91750879-6b28-4d21-9f50-9032c285ffbe None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:17.836517 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91750879-6b28-4d21-9f50-9032c285ffbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:17.878947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG passlib.handlers.bcrypt [None req-91750879-6b28-4d21-9f50-9032c285ffbe None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80160) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 07:23:17.879402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG passlib.handlers.bcrypt [None req-91750879-6b28-4d21-9f50-9032c285ffbe None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80160) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 07:23:17.889559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:17.912441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb31857-c548-4442-a869-f53c9b7fae2e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:17.925090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-91750879-6b28-4d21-9f50-9032c285ffbe None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:17.926727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-91750879-6b28-4d21-9f50-9032c285ffbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:17.936960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:18.021431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb31857-c548-4442-a869-f53c9b7fae2e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:18.029165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:18.050796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91750879-6b28-4d21-9f50-9032c285ffbe None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:18.054446 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dcff4-c4b3-47f9-bbde-173789525c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:18.055134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dcff4-c4b3-47f9-bbde-173789525c17 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:18.055428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dcff4-c4b3-47f9-bbde-173789525c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:18.055713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dcff4-c4b3-47f9-bbde-173789525c17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:18.067482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384dcff4-c4b3-47f9-bbde-173789525c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-384dcff4-c4b3-47f9-bbde-173789525c17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:18.076297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:18.078006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:23:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:18.083877 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dcff4-c4b3-47f9-bbde-173789525c17 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:18.153095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91750879-6b28-4d21-9f50-9032c285ffbe None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:18.155415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:18.164380 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:18.182609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dcff4-c4b3-47f9-bbde-173789525c17 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:18.191446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f164437-5efc-4548-ab21-0cc6370f8adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:18.192307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f164437-5efc-4548-ab21-0cc6370f8adf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:18.192590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f164437-5efc-4548-ab21-0cc6370f8adf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:18.192866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f164437-5efc-4548-ab21-0cc6370f8adf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:18.216084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6a837-be29-45b3-9ae3-c46138df86eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:18.216694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6a837-be29-45b3-9ae3-c46138df86eb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:18.216974 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6a837-be29-45b3-9ae3-c46138df86eb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:18.217251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6a837-be29-45b3-9ae3-c46138df86eb None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:18.218517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6a837-be29-45b3-9ae3-c46138df86eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:18.218819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6a837-be29-45b3-9ae3-c46138df86eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:18.220250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6a837-be29-45b3-9ae3-c46138df86eb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:18.233116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:18.238747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6a837-be29-45b3-9ae3-c46138df86eb None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:18.259972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f164437-5efc-4548-ab21-0cc6370f8adf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:18.261666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f164437-5efc-4548-ab21-0cc6370f8adf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:18.366332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:18.371943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f164437-5efc-4548-ab21-0cc6370f8adf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:18.375292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:18.443794 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6a837-be29-45b3-9ae3-c46138df86eb None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:18.470961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f164437-5efc-4548-ab21-0cc6370f8adf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:18.478146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da088405-4696-4234-b9ae-91beb399345d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:18.478787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da088405-4696-4234-b9ae-91beb399345d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:18.479088 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da088405-4696-4234-b9ae-91beb399345d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:18.479377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da088405-4696-4234-b9ae-91beb399345d None admin] PATH_INFO: `/v3/users/aae3d76a87c24d97b28248f570423938` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:18.480531 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da088405-4696-4234-b9ae-91beb399345d None admin] RBAC: Authorizing `identity:get_user(user_id=aae3d76a87c24d97b28248f570423938)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:18.481985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da088405-4696-4234-b9ae-91beb399345d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:18.483648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:18 2022] GET /identity/v3/users/aae3d76a87c24d97b28248f570423938 => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:18.488988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da088405-4696-4234-b9ae-91beb399345d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:18.503617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:18.506202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20466a-4a40-49ad-8b7b-a60ea089e20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:18.507091 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20466a-4a40-49ad-8b7b-a60ea089e20a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:18.507389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20466a-4a40-49ad-8b7b-a60ea089e20a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:18.507666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20466a-4a40-49ad-8b7b-a60ea089e20a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:18.526365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20466a-4a40-49ad-8b7b-a60ea089e20a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:18.527854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20466a-4a40-49ad-8b7b-a60ea089e20a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-bf20466a-4a40-49ad-8b7b-a60ea089e20a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:23:18.545966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:18.546845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:18.547680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:23:18.547680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:23:18.547680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:23:18.547680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:18.547680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 9/18] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:23:18.552985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20466a-4a40-49ad-8b7b-a60ea089e20a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:18.589139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da088405-4696-4234-b9ae-91beb399345d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:18.607481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:18.624174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd067ae2-98a4-47d2-8c77-17c33aa65ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:18.624753 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd067ae2-98a4-47d2-8c77-17c33aa65ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:18.625038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd067ae2-98a4-47d2-8c77-17c33aa65ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:18.625319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd067ae2-98a4-47d2-8c77-17c33aa65ae8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:18.626470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd067ae2-98a4-47d2-8c77-17c33aa65ae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:18.627954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd067ae2-98a4-47d2-8c77-17c33aa65ae8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:18.629531 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 10/19] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:18.635100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd067ae2-98a4-47d2-8c77-17c33aa65ae8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:18.652531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20466a-4a40-49ad-8b7b-a60ea089e20a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:18.685479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820cd96-6b21-4d1b-a7bd-b8f3008783d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:18.686096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820cd96-6b21-4d1b-a7bd-b8f3008783d3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:18.686379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820cd96-6b21-4d1b-a7bd-b8f3008783d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:18.686665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820cd96-6b21-4d1b-a7bd-b8f3008783d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:18.687455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820cd96-6b21-4d1b-a7bd-b8f3008783d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:18.687760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820cd96-6b21-4d1b-a7bd-b8f3008783d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:18.689150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820cd96-6b21-4d1b-a7bd-b8f3008783d3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:18.706456 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 10/20] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:18.718603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:18.739896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd067ae2-98a4-47d2-8c77-17c33aa65ae8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:18.777307 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c638a-b037-4e33-aaca-e3592a3d7414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:18.777945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c638a-b037-4e33-aaca-e3592a3d7414 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:18.778237 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c638a-b037-4e33-aaca-e3592a3d7414 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:18.778522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c638a-b037-4e33-aaca-e3592a3d7414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:18.779449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c638a-b037-4e33-aaca-e3592a3d7414 None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.domain.id=default, user.id=aae3d76a87c24d97b28248f570423938) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:18.779773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c638a-b037-4e33-aaca-e3592a3d7414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:18.781929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c638a-b037-4e33-aaca-e3592a3d7414 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:18.789559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 11/21] 10.222.0.40 () {56 vars in 1322 bytes} [Mon Aug 1 07:23:18 2022] GET /identity/v3/role_assignments?user.id=aae3d76a87c24d97b28248f570423938&scope.domain.id=default&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:21.178660 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e447777-80db-4715-a5c5-608c18c57431 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:21.179520 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e447777-80db-4715-a5c5-608c18c57431 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:21.179942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e447777-80db-4715-a5c5-608c18c57431 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:21.181600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 11/22] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:23:21.186889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3d74f-cf3a-487d-8e17-e3c8e3093680 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:21.187673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3d74f-cf3a-487d-8e17-e3c8e3093680 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:21.188101 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3d74f-cf3a-487d-8e17-e3c8e3093680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:21.216126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-98e3d74f-cf3a-487d-8e17-e3c8e3093680 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:21.218059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-98e3d74f-cf3a-487d-8e17-e3c8e3093680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:21.292215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a67467-f62a-4a34-a4ca-ab7811120e78 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:21.292521 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a67467-f62a-4a34-a4ca-ab7811120e78 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:21.292766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a67467-f62a-4a34-a4ca-ab7811120e78 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:21.294344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 12/23] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:23:21.300235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4f2e7-14aa-4a7e-be9b-e77a7efce8bf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:21.300776 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4f2e7-14aa-4a7e-be9b-e77a7efce8bf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:21.301255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4f2e7-14aa-4a7e-be9b-e77a7efce8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:21.303879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 12/24] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:21.337766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3d74f-cf3a-487d-8e17-e3c8e3093680 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:21.340100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-27a4f2e7-14aa-4a7e-be9b-e77a7efce8bf None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:21.342261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-27a4f2e7-14aa-4a7e-be9b-e77a7efce8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:21.467562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:21.481272 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 13/25] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:21.515339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4f2e7-14aa-4a7e-be9b-e77a7efce8bf None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:21.526904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3d74f-cf3a-487d-8e17-e3c8e3093680 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:21.609541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:21.624992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cffaa-5139-4f79-adb9-efef54bc6adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:21.625589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cffaa-5139-4f79-adb9-efef54bc6adc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:21.625876 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cffaa-5139-4f79-adb9-efef54bc6adc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:21.626157 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cffaa-5139-4f79-adb9-efef54bc6adc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:21.660041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cffaa-5139-4f79-adb9-efef54bc6adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-bc1cffaa-5139-4f79-adb9-efef54bc6adc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:21.665077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:21.666091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:21.667046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:21.667046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:21.667046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:21.667046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:21.667046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:21.667046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:21.667046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:21.667046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:21.667046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:21.667046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:21.667046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:21.667046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:21.671423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 13/26] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:23:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:23:21.676217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cffaa-5139-4f79-adb9-efef54bc6adc None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:21.735584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4f2e7-14aa-4a7e-be9b-e77a7efce8bf None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:21.807558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0c390-7602-4412-8401-70b3b594ad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:21.808400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0c390-7602-4412-8401-70b3b594ad86 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:21.808694 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0c390-7602-4412-8401-70b3b594ad86 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:21.808967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0c390-7602-4412-8401-70b3b594ad86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:21.834492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0c390-7602-4412-8401-70b3b594ad86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:21.836076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0c390-7602-4412-8401-70b3b594ad86 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:21.854157 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cffaa-5139-4f79-adb9-efef54bc6adc None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-22b0c390-7602-4412-8401-70b3b594ad86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:23:21.862495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:21.863519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:21.864441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:23:21.864441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:23:21.864441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:23:21.864441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:21.864441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 14/27] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:23:21.864861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dce9ed-fe92-427b-b8ee-7dd8e3405fe8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:21.865144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dce9ed-fe92-427b-b8ee-7dd8e3405fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:21.865419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dce9ed-fe92-427b-b8ee-7dd8e3405fe8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:21.871605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 15/28] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:21.877875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187065-beb7-4fa5-85c4-d5f8e62911f9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:21.878168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187065-beb7-4fa5-85c4-d5f8e62911f9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:21.878448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187065-beb7-4fa5-85c4-d5f8e62911f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:21.902477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d612e5d-20cf-4ef0-9838-e059ea985974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:21.903175 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d612e5d-20cf-4ef0-9838-e059ea985974 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:21.903461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d612e5d-20cf-4ef0-9838-e059ea985974 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:21.903743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d612e5d-20cf-4ef0-9838-e059ea985974 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:21.904467 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d612e5d-20cf-4ef0-9838-e059ea985974 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:21.904774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d612e5d-20cf-4ef0-9838-e059ea985974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:21.906787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d612e5d-20cf-4ef0-9838-e059ea985974 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:21.912356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:21.913641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c3187065-beb7-4fa5-85c4-d5f8e62911f9 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:21.914832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c3187065-beb7-4fa5-85c4-d5f8e62911f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:21.927768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 14/29] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:21.928658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d612e5d-20cf-4ef0-9838-e059ea985974 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:22.053461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d612e5d-20cf-4ef0-9838-e059ea985974 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:22.062693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 16/30] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:22.063586 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187065-beb7-4fa5-85c4-d5f8e62911f9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:22.094784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d384f6-8005-481c-9623-3ae1b5b49d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:22.095784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d384f6-8005-481c-9623-3ae1b5b49d60 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:22.096268 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d384f6-8005-481c-9623-3ae1b5b49d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:22.096749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d384f6-8005-481c-9623-3ae1b5b49d60 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:22.097721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d384f6-8005-481c-9623-3ae1b5b49d60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:22.098251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d384f6-8005-481c-9623-3ae1b5b49d60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:22.099981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d384f6-8005-481c-9623-3ae1b5b49d60 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:22.107127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:22.119818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 15/31] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:22.121325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e50204-88b2-4b02-914c-c8a2cc3ae824 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:22.121782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e50204-88b2-4b02-914c-c8a2cc3ae824 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:22.122221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e50204-88b2-4b02-914c-c8a2cc3ae824 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:22.124357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 16/32] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:22.126701 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfa409-51d8-4b72-84ce-39164bd06385 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:22.127176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfa409-51d8-4b72-84ce-39164bd06385 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:22.127602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfa409-51d8-4b72-84ce-39164bd06385 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:22.129123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 17/33] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:22.130002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfa409-51d8-4b72-84ce-39164bd06385 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:22.197722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:22.251480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfa409-51d8-4b72-84ce-39164bd06385 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:22.286844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ea08d-6c3d-4bba-a566-6c51a2ec005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:22.287509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ea08d-6c3d-4bba-a566-6c51a2ec005e None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:22.287801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ea08d-6c3d-4bba-a566-6c51a2ec005e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:22.288084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ea08d-6c3d-4bba-a566-6c51a2ec005e None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:22.288808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ea08d-6c3d-4bba-a566-6c51a2ec005e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:22.290220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ea08d-6c3d-4bba-a566-6c51a2ec005e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:22.291961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187065-beb7-4fa5-85c4-d5f8e62911f9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:22.329404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a5e88-54fe-4bb4-9e95-4fb94596ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:22.329977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a5e88-54fe-4bb4-9e95-4fb94596ecf9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:22.330262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a5e88-54fe-4bb4-9e95-4fb94596ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:22.330548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a5e88-54fe-4bb4-9e95-4fb94596ecf9 None admin] PATH_INFO: `/v3/users/aae3d76a87c24d97b28248f570423938` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:22.332104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a5e88-54fe-4bb4-9e95-4fb94596ecf9 None admin] RBAC: Authorizing `identity:get_user(user_id=aae3d76a87c24d97b28248f570423938)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:22.333541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a5e88-54fe-4bb4-9e95-4fb94596ecf9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:22.335159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 17/34] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:22 2022] GET /identity/v3/users/aae3d76a87c24d97b28248f570423938 => generated 247 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:22.340240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a5e88-54fe-4bb4-9e95-4fb94596ecf9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:22.412861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 18/35] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Aug 1 07:23:22 2022] POST /identity/v3/roles => generated 219 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:23:22.413624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88522270-04cf-4d0d-82e3-f732c74c68fb None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:22.413918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88522270-04cf-4d0d-82e3-f732c74c68fb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:22.414196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88522270-04cf-4d0d-82e3-f732c74c68fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:22.438943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-88522270-04cf-4d0d-82e3-f732c74c68fb None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:22.441465 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-88522270-04cf-4d0d-82e3-f732c74c68fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:22.452493 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:22.462672 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a5e88-54fe-4bb4-9e95-4fb94596ecf9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:22.503952 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e1f2f-b01a-4ce6-85e3-4d7bd080de40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:22.504585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e1f2f-b01a-4ce6-85e3-4d7bd080de40 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:22.504875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e1f2f-b01a-4ce6-85e3-4d7bd080de40 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:22.505158 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e1f2f-b01a-4ce6-85e3-4d7bd080de40 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:22.505851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e1f2f-b01a-4ce6-85e3-4d7bd080de40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:22.507552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e1f2f-b01a-4ce6-85e3-4d7bd080de40 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:22.538738 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 19/36] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:22.541173 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364c344-da74-4919-ad85-e3fefb39b218 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:22.541693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364c344-da74-4919-ad85-e3fefb39b218 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:22.542481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364c344-da74-4919-ad85-e3fefb39b218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:22.563009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d364c344-da74-4919-ad85-e3fefb39b218 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:22.564663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d364c344-da74-4919-ad85-e3fefb39b218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:22.617682 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:22.647973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 20/37] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:22.650078 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364c344-da74-4919-ad85-e3fefb39b218 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:22.663344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 18/38] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:23:22 2022] POST /identity/v3/projects => generated 301 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:23:22.664303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e1f2f-b01a-4ce6-85e3-4d7bd080de40 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:22.779412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e1f2f-b01a-4ce6-85e3-4d7bd080de40 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:22.790721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364c344-da74-4919-ad85-e3fefb39b218 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:22.819500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffd9a9-c626-4f88-8518-a0caf278e960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:22.820140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffd9a9-c626-4f88-8518-a0caf278e960 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:22.820429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffd9a9-c626-4f88-8518-a0caf278e960 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:22.821448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffd9a9-c626-4f88-8518-a0caf278e960 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:22.821448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffd9a9-c626-4f88-8518-a0caf278e960 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:22.822934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffd9a9-c626-4f88-8518-a0caf278e960 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:22.833712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d362a-7a1e-4d9a-ab93-5c6606ef2d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:22.834708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d362a-7a1e-4d9a-ab93-5c6606ef2d39 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:22.834903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d362a-7a1e-4d9a-ab93-5c6606ef2d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:22.836260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d362a-7a1e-4d9a-ab93-5c6606ef2d39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:22.836477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d362a-7a1e-4d9a-ab93-5c6606ef2d39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:22.838712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d362a-7a1e-4d9a-ab93-5c6606ef2d39 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:22.840271 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 19/39] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Aug 1 07:23:22 2022] POST /identity/v3/roles => generated 225 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:23:22.840377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 21/40] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:22.841929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d362a-7a1e-4d9a-ab93-5c6606ef2d39 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:22.848631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdc12b-274c-4062-a920-2f8ee17b0248 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:22.849289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdc12b-274c-4062-a920-2f8ee17b0248 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:22.849777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdc12b-274c-4062-a920-2f8ee17b0248 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:22.851828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 20/41] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:22.852813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdc12b-274c-4062-a920-2f8ee17b0248 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:22.866107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:22.921159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:22.978506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d362a-7a1e-4d9a-ab93-5c6606ef2d39 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:22.997378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdc12b-274c-4062-a920-2f8ee17b0248 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:23.022500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872147a-a0a9-472f-88d8-601f1765c65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:23.023160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872147a-a0a9-472f-88d8-601f1765c65d None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:23.023449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872147a-a0a9-472f-88d8-601f1765c65d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:23.023728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872147a-a0a9-472f-88d8-601f1765c65d None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:23.024447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872147a-a0a9-472f-88d8-601f1765c65d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:23.025836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872147a-a0a9-472f-88d8-601f1765c65d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:23.038312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b7886-6c64-40cd-b121-91958c780dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:23.039060 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b7886-6c64-40cd-b121-91958c780dce None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:23.039378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b7886-6c64-40cd-b121-91958c780dce None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:23.039665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b7886-6c64-40cd-b121-91958c780dce None admin] PATH_INFO: `/v3/domains/default/users/aae3d76a87c24d97b28248f570423938/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:23.041925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b7886-6c64-40cd-b121-91958c780dce None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=aae3d76a87c24d97b28248f570423938, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:23.043250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 22/42] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Aug 1 07:23:22 2022] POST /identity/v3/roles => generated 223 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:23:23.043572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b7886-6c64-40cd-b121-91958c780dce None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:23.044470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782b5e9-cbba-4ee4-82cc-515ae74697b0 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:23.044760 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782b5e9-cbba-4ee4-82cc-515ae74697b0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:23.045040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782b5e9-cbba-4ee4-82cc-515ae74697b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:23.063830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 21/43] 10.222.0.40 () {58 vars in 1361 bytes} [Mon Aug 1 07:23:22 2022] PUT /identity/v3/domains/default/users/aae3d76a87c24d97b28248f570423938/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:23:23.065904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a782b5e9-cbba-4ee4-82cc-515ae74697b0 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:23.067214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a782b5e9-cbba-4ee4-82cc-515ae74697b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:23.154981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 23/44] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:23.178904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b7886-6c64-40cd-b121-91958c780dce None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:23.254006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:23.290318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b7886-6c64-40cd-b121-91958c780dce None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:23.325113 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6005008-44c6-4622-b2fe-e0ad81ed376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:23.325759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6005008-44c6-4622-b2fe-e0ad81ed376c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:23.326048 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6005008-44c6-4622-b2fe-e0ad81ed376c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:23.326331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6005008-44c6-4622-b2fe-e0ad81ed376c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:23.328337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6005008-44c6-4622-b2fe-e0ad81ed376c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:23.329812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6005008-44c6-4622-b2fe-e0ad81ed376c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:23.331449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 22/45] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:23.337308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782b5e9-cbba-4ee4-82cc-515ae74697b0 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:23.371280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:23.438068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782b5e9-cbba-4ee4-82cc-515ae74697b0 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:23.471156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1391d5d-b258-4fab-bf6d-6a8762c4da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:23.471771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1391d5d-b258-4fab-bf6d-6a8762c4da3a None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:23.472056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1391d5d-b258-4fab-bf6d-6a8762c4da3a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:23.472335 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1391d5d-b258-4fab-bf6d-6a8762c4da3a None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:23.473012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1391d5d-b258-4fab-bf6d-6a8762c4da3a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:23.474469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1391d5d-b258-4fab-bf6d-6a8762c4da3a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:23.502006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 24/46] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:23:23 2022] POST /identity/v3/projects => generated 312 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:23:25.108176 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0800c16-ccdf-460b-82e1-dfb33f01526f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:25.108759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0800c16-ccdf-460b-82e1-dfb33f01526f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:25.108759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0800c16-ccdf-460b-82e1-dfb33f01526f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:25.110033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 23/47] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:25.115202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe7bce-c4a7-4ea3-b7a7-22a3979d41c3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:25.115614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe7bce-c4a7-4ea3-b7a7-22a3979d41c3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:25.115921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe7bce-c4a7-4ea3-b7a7-22a3979d41c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:25.134461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a2fe7bce-c4a7-4ea3-b7a7-22a3979d41c3 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:25.135682 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a2fe7bce-c4a7-4ea3-b7a7-22a3979d41c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:25.201832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 25/48] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:25.224443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0800c16-ccdf-460b-82e1-dfb33f01526f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:25.324391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0800c16-ccdf-460b-82e1-dfb33f01526f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:25.357708 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c026a-7500-4095-bca8-e98e9a4637b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:25.358372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c026a-7500-4095-bca8-e98e9a4637b6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:25.358637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c026a-7500-4095-bca8-e98e9a4637b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:25.358923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c026a-7500-4095-bca8-e98e9a4637b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:25.362495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:25.371261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c026a-7500-4095-bca8-e98e9a4637b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7f1c026a-7500-4095-bca8-e98e9a4637b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:25.377739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:25.378559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:25.379613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 24/49] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:23:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:25.384735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe7bce-c4a7-4ea3-b7a7-22a3979d41c3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:25.403147 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:25.475592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57cc16-1b32-4f62-be0b-82d7d64d7f78 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:25.475801 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57cc16-1b32-4f62-be0b-82d7d64d7f78 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:25.476087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57cc16-1b32-4f62-be0b-82d7d64d7f78 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:25.477431 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 25/50] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:23:25.482574 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bea77a-e618-412f-a2cd-97216cba29a3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:25.482919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bea77a-e618-412f-a2cd-97216cba29a3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:25.483343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bea77a-e618-412f-a2cd-97216cba29a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:25.485532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe7bce-c4a7-4ea3-b7a7-22a3979d41c3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:25.501150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-14bea77a-e618-412f-a2cd-97216cba29a3 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:25.502675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-14bea77a-e618-412f-a2cd-97216cba29a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:25.520265 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e166056-4f68-4b6e-8076-86cdf379d786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:25.520888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e166056-4f68-4b6e-8076-86cdf379d786 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:25.521174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e166056-4f68-4b6e-8076-86cdf379d786 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:25.521460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e166056-4f68-4b6e-8076-86cdf379d786 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:25.522168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e166056-4f68-4b6e-8076-86cdf379d786 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:25.522476 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e166056-4f68-4b6e-8076-86cdf379d786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:25.523896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e166056-4f68-4b6e-8076-86cdf379d786 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:25.533430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 26/51] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:25.538878 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e166056-4f68-4b6e-8076-86cdf379d786 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:25.570158 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 26/52] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:25.592855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bea77a-e618-412f-a2cd-97216cba29a3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:25.620110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:25.642536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e166056-4f68-4b6e-8076-86cdf379d786 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:25.678081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d73d0a-c3ca-4eee-a383-58a3fae5bd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:25.678730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73d0a-c3ca-4eee-a383-58a3fae5bd73 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:25.679019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73d0a-c3ca-4eee-a383-58a3fae5bd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:25.679574 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73d0a-c3ca-4eee-a383-58a3fae5bd73 None admin] PATH_INFO: `/v3/users/aae3d76a87c24d97b28248f570423938` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:25.680523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d73d0a-c3ca-4eee-a383-58a3fae5bd73 None admin] RBAC: Authorizing `identity:get_user(user_id=aae3d76a87c24d97b28248f570423938)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:25.682002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d73d0a-c3ca-4eee-a383-58a3fae5bd73 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:25.683645 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 27/53] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:25 2022] GET /identity/v3/users/aae3d76a87c24d97b28248f570423938 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:25.689159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d73d0a-c3ca-4eee-a383-58a3fae5bd73 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:25.694034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bea77a-e618-412f-a2cd-97216cba29a3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:25.720962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:25.730439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d62486-e595-4082-b949-517808b3c23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:25.731022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d62486-e595-4082-b949-517808b3c23d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:25.731337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d62486-e595-4082-b949-517808b3c23d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:25.731619 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d62486-e595-4082-b949-517808b3c23d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:25.732822 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d62486-e595-4082-b949-517808b3c23d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:25.734798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d62486-e595-4082-b949-517808b3c23d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:25.736421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 27/54] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:25.741825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d62486-e595-4082-b949-517808b3c23d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:25.789161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d73d0a-c3ca-4eee-a383-58a3fae5bd73 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:25.823538 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627ca53-8874-4745-9a79-5669374b1fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:25.824187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627ca53-8874-4745-9a79-5669374b1fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:25.824473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627ca53-8874-4745-9a79-5669374b1fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:25.824760 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627ca53-8874-4745-9a79-5669374b1fc7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:25.825912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627ca53-8874-4745-9a79-5669374b1fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:25.827968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627ca53-8874-4745-9a79-5669374b1fc7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:25.829561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 28/55] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:25.835228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627ca53-8874-4745-9a79-5669374b1fc7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:25.839219 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d62486-e595-4082-b949-517808b3c23d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:25.850597 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:25.874477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded928fa-3e25-4f85-80a2-ef1be9cbc536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:25.874878 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded928fa-3e25-4f85-80a2-ef1be9cbc536 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:25.875199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded928fa-3e25-4f85-80a2-ef1be9cbc536 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:25.875481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded928fa-3e25-4f85-80a2-ef1be9cbc536 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:25.876161 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded928fa-3e25-4f85-80a2-ef1be9cbc536 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:25.877608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded928fa-3e25-4f85-80a2-ef1be9cbc536 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:25.903989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 28/56] 10.222.0.40 () {60 vars in 1140 bytes} [Mon Aug 1 07:23:25 2022] POST /identity/v3/projects => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:23:25.941489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627ca53-8874-4745-9a79-5669374b1fc7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:25.978262 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a1b51-26f9-4c19-9a27-97158794dffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:25.978901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1b51-26f9-4c19-9a27-97158794dffb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:25.979222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1b51-26f9-4c19-9a27-97158794dffb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:25.979508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1b51-26f9-4c19-9a27-97158794dffb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:25.980311 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a1b51-26f9-4c19-9a27-97158794dffb None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.domain.id=default, user.id=aae3d76a87c24d97b28248f570423938) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:25.980681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a1b51-26f9-4c19-9a27-97158794dffb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:25.982110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a1b51-26f9-4c19-9a27-97158794dffb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:25.989649 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 29/57] 10.222.0.40 () {56 vars in 1322 bytes} [Mon Aug 1 07:23:25 2022] GET /identity/v3/role_assignments?user.id=aae3d76a87c24d97b28248f570423938&scope.domain.id=default&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:27.900320 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8b7a1-8f98-4124-a59a-ba2914be853e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:27.900843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8b7a1-8f98-4124-a59a-ba2914be853e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:27.900933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8b7a1-8f98-4124-a59a-ba2914be853e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:27.902234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 29/58] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:27.908271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f5209-ce93-4433-ac21-d06389e369ef None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:27.908736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f5209-ce93-4433-ac21-d06389e369ef None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:27.908849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f5209-ce93-4433-ac21-d06389e369ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:27.927177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4c8f5209-ce93-4433-ac21-d06389e369ef None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:27.928384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4c8f5209-ce93-4433-ac21-d06389e369ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:27.995478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 30/59] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:28.020292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8b7a1-8f98-4124-a59a-ba2914be853e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:28.066834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:28.119278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8b7a1-8f98-4124-a59a-ba2914be853e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:28.152653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b55ef-bf7a-44da-b79d-38bda7c4fde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:28.153224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b55ef-bf7a-44da-b79d-38bda7c4fde3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:28.153513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b55ef-bf7a-44da-b79d-38bda7c4fde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:28.153795 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b55ef-bf7a-44da-b79d-38bda7c4fde3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:28.154969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b55ef-bf7a-44da-b79d-38bda7c4fde3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:28.156434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b55ef-bf7a-44da-b79d-38bda7c4fde3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:28.158002 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 30/60] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:28.163438 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f5209-ce93-4433-ac21-d06389e369ef None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:28.173750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:28.259391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f5209-ce93-4433-ac21-d06389e369ef None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:28.291701 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57bfe9-dfee-43c0-a3cb-84e92407a1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:28.292305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57bfe9-dfee-43c0-a3cb-84e92407a1a8 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:28.292589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57bfe9-dfee-43c0-a3cb-84e92407a1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:28.292868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57bfe9-dfee-43c0-a3cb-84e92407a1a8 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:28.293547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57bfe9-dfee-43c0-a3cb-84e92407a1a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:28.294984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57bfe9-dfee-43c0-a3cb-84e92407a1a8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:28.321059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 31/61] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:23:28 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:23:31.300955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7d86b-752c-4e6a-ac65-502a3eb01ff0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:31.301925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7d86b-752c-4e6a-ac65-502a3eb01ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:31.302423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7d86b-752c-4e6a-ac65-502a3eb01ff0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:31.304165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 31/62] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:31.309585 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ce687-f932-49a6-bb5e-7b16137d84ce None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:31.310379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ce687-f932-49a6-bb5e-7b16137d84ce None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:31.311523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ce687-f932-49a6-bb5e-7b16137d84ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:31.340111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c17ce687-f932-49a6-bb5e-7b16137d84ce None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:31.341703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c17ce687-f932-49a6-bb5e-7b16137d84ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:31.420682 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:31.438001 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 32/63] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:31.468753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7d86b-752c-4e6a-ac65-502a3eb01ff0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:31.563804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3c833-e0a7-48e9-95fd-62f7cbc895cc None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:31.564450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3c833-e0a7-48e9-95fd-62f7cbc895cc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:31.564925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3c833-e0a7-48e9-95fd-62f7cbc895cc None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:31.566568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 33/64] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:31.571792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069094c-e3e9-4fc9-b4e3-2c7f28f5c167 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:31.572302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069094c-e3e9-4fc9-b4e3-2c7f28f5c167 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:31.572806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069094c-e3e9-4fc9-b4e3-2c7f28f5c167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:31.605824 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f069094c-e3e9-4fc9-b4e3-2c7f28f5c167 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:31.610317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f069094c-e3e9-4fc9-b4e3-2c7f28f5c167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:31.629773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7d86b-752c-4e6a-ac65-502a3eb01ff0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:31.659440 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:31.694131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c684d-24ec-4fe1-ad49-1271609fb2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:31.694799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c684d-24ec-4fe1-ad49-1271609fb2ed None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:31.695099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c684d-24ec-4fe1-ad49-1271609fb2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:31.695391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c684d-24ec-4fe1-ad49-1271609fb2ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:31.720368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 34/65] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:31.727795 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c684d-24ec-4fe1-ad49-1271609fb2ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a37c684d-24ec-4fe1-ad49-1271609fb2ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:31.733722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:31.734566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:31.735350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:31.735350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:31.735350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:31.735350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:31.735350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:31.735350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:31.735350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:31.735350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:31.735350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:31.735350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:31.735350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:31.735350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:31.737725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 32/66] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:31.744393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069094c-e3e9-4fc9-b4e3-2c7f28f5c167 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:31.754894 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c684d-24ec-4fe1-ad49-1271609fb2ed None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:31.890747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069094c-e3e9-4fc9-b4e3-2c7f28f5c167 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:31.916778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:31.929075 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58008ee6-8e3f-41fc-ad68-b12b66682d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:31.929917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58008ee6-8e3f-41fc-ad68-b12b66682d88 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:31.930347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58008ee6-8e3f-41fc-ad68-b12b66682d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:31.930768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58008ee6-8e3f-41fc-ad68-b12b66682d88 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:31.931238 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c684d-24ec-4fe1-ad49-1271609fb2ed None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:31.931853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58008ee6-8e3f-41fc-ad68-b12b66682d88 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:31.932309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58008ee6-8e3f-41fc-ad68-b12b66682d88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:31.933847 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58008ee6-8e3f-41fc-ad68-b12b66682d88 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:31.943344 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 35/67] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:31.944824 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07fcdb-8f39-45f3-af60-62d655f5cf05 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:31.950611 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07fcdb-8f39-45f3-af60-62d655f5cf05 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:31.951662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07fcdb-8f39-45f3-af60-62d655f5cf05 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:31.957968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 36/68] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:31.957968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07fcdb-8f39-45f3-af60-62d655f5cf05 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:31.974326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67264cd-f912-426a-87c7-aa9f1e272ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:31.975019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67264cd-f912-426a-87c7-aa9f1e272ede None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:31.975342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67264cd-f912-426a-87c7-aa9f1e272ede None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:31.975623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67264cd-f912-426a-87c7-aa9f1e272ede None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:31.988081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67264cd-f912-426a-87c7-aa9f1e272ede None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a67264cd-f912-426a-87c7-aa9f1e272ede None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:31.992961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:31.993781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:31.994535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 33/69] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:23:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:23:31.995274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945393c-74b3-4bd5-8de4-5620344c9864 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:31.995702 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945393c-74b3-4bd5-8de4-5620344c9864 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:31.996139 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945393c-74b3-4bd5-8de4-5620344c9864 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:31.997523 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 34/70] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:31.998710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6276a-85e8-4ad0-a606-06cdd80b84d6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:31.999174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6276a-85e8-4ad0-a606-06cdd80b84d6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:31.999292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6276a-85e8-4ad0-a606-06cdd80b84d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.021102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3db6276a-85e8-4ad0-a606-06cdd80b84d6 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:32.022325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3db6276a-85e8-4ad0-a606-06cdd80b84d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:32.049462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:32.076878 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07fcdb-8f39-45f3-af60-62d655f5cf05 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:32.090345 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 35/71] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:32.091188 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6276a-85e8-4ad0-a606-06cdd80b84d6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:32.110958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49d465-996c-4dbf-a865-37e5e2f7595c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:32.111698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49d465-996c-4dbf-a865-37e5e2f7595c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49d465-996c-4dbf-a865-37e5e2f7595c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49d465-996c-4dbf-a865-37e5e2f7595c None admin] PATH_INFO: `/v3/users/5b4524f7f2f6410e983dadf2d798c568` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.132704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49d465-996c-4dbf-a865-37e5e2f7595c None admin] RBAC: Authorizing `identity:get_user(user_id=5b4524f7f2f6410e983dadf2d798c568)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:32.134179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49d465-996c-4dbf-a865-37e5e2f7595c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:32.135887 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 37/72] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:31 2022] GET /identity/v3/users/5b4524f7f2f6410e983dadf2d798c568 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:32.137293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b5e50-16eb-4af8-9742-a596f8ed0df2 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.137589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b5e50-16eb-4af8-9742-a596f8ed0df2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.137874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b5e50-16eb-4af8-9742-a596f8ed0df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.156120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-403b5e50-16eb-4af8-9742-a596f8ed0df2 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:32.157239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-403b5e50-16eb-4af8-9742-a596f8ed0df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:32.190418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6276a-85e8-4ad0-a606-06cdd80b84d6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:32.197142 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:32.224678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc6dd7-4746-4a9e-98f3-09421eebfa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:32.225298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc6dd7-4746-4a9e-98f3-09421eebfa9f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.225582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc6dd7-4746-4a9e-98f3-09421eebfa9f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.225763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 38/73] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:32.225854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc6dd7-4746-4a9e-98f3-09421eebfa9f None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.226556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc6dd7-4746-4a9e-98f3-09421eebfa9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:32.226934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc6dd7-4746-4a9e-98f3-09421eebfa9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:32.227022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65816b2b-54b6-4788-8c18-f26c4a7a8a11 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.227305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65816b2b-54b6-4788-8c18-f26c4a7a8a11 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.227596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65816b2b-54b6-4788-8c18-f26c4a7a8a11 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.228356 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc6dd7-4746-4a9e-98f3-09421eebfa9f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:32.229015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 39/74] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:32.229562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65816b2b-54b6-4788-8c18-f26c4a7a8a11 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:32.247982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 36/75] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:32.249353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc6dd7-4746-4a9e-98f3-09421eebfa9f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:32.278682 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:32.330816 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65816b2b-54b6-4788-8c18-f26c4a7a8a11 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:32.351330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc6dd7-4746-4a9e-98f3-09421eebfa9f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:32.364759 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9f3d8-9830-4901-a471-5ead98cdce07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:32.365405 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9f3d8-9830-4901-a471-5ead98cdce07 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.365697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9f3d8-9830-4901-a471-5ead98cdce07 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.365975 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9f3d8-9830-4901-a471-5ead98cdce07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.376522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9f3d8-9830-4901-a471-5ead98cdce07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-50b9f3d8-9830-4901-a471-5ead98cdce07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:32.381482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:32.382329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:32.383099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 40/76] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:32.384903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f397c-68c2-4766-a088-3d1e3e160399 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.385202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f397c-68c2-4766-a088-3d1e3e160399 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.385487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f397c-68c2-4766-a088-3d1e3e160399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.389530 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45f141-6948-4a74-9fec-d990bc6f85bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:32.390165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45f141-6948-4a74-9fec-d990bc6f85bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.390450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45f141-6948-4a74-9fec-d990bc6f85bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.390724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45f141-6948-4a74-9fec-d990bc6f85bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.404418 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-065f397c-68c2-4766-a088-3d1e3e160399 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:32.405640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-065f397c-68c2-4766-a088-3d1e3e160399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:32.431628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:32.457097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45f141-6948-4a74-9fec-d990bc6f85bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:32.458554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45f141-6948-4a74-9fec-d990bc6f85bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:32.472766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 41/77] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:32.473668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f397c-68c2-4766-a088-3d1e3e160399 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:32.487099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:32.562649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 37/78] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:32.563366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45f141-6948-4a74-9fec-d990bc6f85bb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:32.573560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f397c-68c2-4766-a088-3d1e3e160399 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:32.608787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059382b7-5ff1-4fce-9822-5de09ed9d5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:32.609430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059382b7-5ff1-4fce-9822-5de09ed9d5ca None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.609723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059382b7-5ff1-4fce-9822-5de09ed9d5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.610005 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059382b7-5ff1-4fce-9822-5de09ed9d5ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.620138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059382b7-5ff1-4fce-9822-5de09ed9d5ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-059382b7-5ff1-4fce-9822-5de09ed9d5ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:32.625472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:32.626319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:32.627179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 42/79] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:23:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:32.627801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059382b7-5ff1-4fce-9822-5de09ed9d5ca None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:32.652914 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:32.665035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45f141-6948-4a74-9fec-d990bc6f85bb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:32.690755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:32.700050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849af03-11ba-45b4-9924-65c74ca3f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:32.700691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849af03-11ba-45b4-9924-65c74ca3f155 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.701545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849af03-11ba-45b4-9924-65c74ca3f155 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.702211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849af03-11ba-45b4-9924-65c74ca3f155 None admin] PATH_INFO: `/v3/users/aae3d76a87c24d97b28248f570423938` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.703015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849af03-11ba-45b4-9924-65c74ca3f155 None admin] RBAC: Authorizing `identity:get_user(user_id=aae3d76a87c24d97b28248f570423938)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:32.704497 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849af03-11ba-45b4-9924-65c74ca3f155 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:32.706115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 38/80] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:32 2022] GET /identity/v3/users/aae3d76a87c24d97b28248f570423938 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:32.710926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849af03-11ba-45b4-9924-65c74ca3f155 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:32.735870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059382b7-5ff1-4fce-9822-5de09ed9d5ca None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:32.769991 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757f337-e88b-4868-bf31-2ce1f99bb21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:32.770606 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757f337-e88b-4868-bf31-2ce1f99bb21d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.770891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757f337-e88b-4868-bf31-2ce1f99bb21d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.771203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757f337-e88b-4868-bf31-2ce1f99bb21d None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.771913 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757f337-e88b-4868-bf31-2ce1f99bb21d None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:32.772234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757f337-e88b-4868-bf31-2ce1f99bb21d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:32.773614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757f337-e88b-4868-bf31-2ce1f99bb21d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:32.782288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 43/81] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:32.783712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757f337-e88b-4868-bf31-2ce1f99bb21d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:32.807274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849af03-11ba-45b4-9924-65c74ca3f155 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:32.841443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d504530-51e5-4a52-9d09-21d23a1395bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:32.842766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d504530-51e5-4a52-9d09-21d23a1395bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.843086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d504530-51e5-4a52-9d09-21d23a1395bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.843396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d504530-51e5-4a52-9d09-21d23a1395bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.844556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d504530-51e5-4a52-9d09-21d23a1395bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:32.846023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d504530-51e5-4a52-9d09-21d23a1395bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:32.847799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 39/82] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:32.848405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d504530-51e5-4a52-9d09-21d23a1395bb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:32.871352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:32.887240 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757f337-e88b-4868-bf31-2ce1f99bb21d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:32.909084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:32.921866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9b3b3-4d8c-46e3-a329-159e60aefa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:32.922505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9b3b3-4d8c-46e3-a329-159e60aefa01 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.922790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9b3b3-4d8c-46e3-a329-159e60aefa01 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.923096 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9b3b3-4d8c-46e3-a329-159e60aefa01 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.924250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9b3b3-4d8c-46e3-a329-159e60aefa01 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:32.925686 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9b3b3-4d8c-46e3-a329-159e60aefa01 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:32.927366 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 44/83] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:32 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:32.928118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9b3b3-4d8c-46e3-a329-159e60aefa01 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:32.948811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d504530-51e5-4a52-9d09-21d23a1395bb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:32.984201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5134d1a-923d-458b-a871-4b5a1143de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:32.984819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5134d1a-923d-458b-a871-4b5a1143de3f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:32.985107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5134d1a-923d-458b-a871-4b5a1143de3f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:32.985387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5134d1a-923d-458b-a871-4b5a1143de3f None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:32.986095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5134d1a-923d-458b-a871-4b5a1143de3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:32.986403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5134d1a-923d-458b-a871-4b5a1143de3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:32.988908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5134d1a-923d-458b-a871-4b5a1143de3f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:32.995799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 40/84] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:23:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:32.996704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5134d1a-923d-458b-a871-4b5a1143de3f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:33.036483 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9b3b3-4d8c-46e3-a329-159e60aefa01 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:33.069270 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6512a8c-ec57-40c1-927e-b0a87165104d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:33.069878 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6512a8c-ec57-40c1-927e-b0a87165104d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:33.070162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6512a8c-ec57-40c1-927e-b0a87165104d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:33.070440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6512a8c-ec57-40c1-927e-b0a87165104d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:33.094245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:33.096455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5134d1a-923d-458b-a871-4b5a1143de3f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:33.120061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:33.131429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d4817-775d-4bb2-b8ff-e1a464bcccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:33.132065 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d4817-775d-4bb2-b8ff-e1a464bcccd1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:33.132359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d4817-775d-4bb2-b8ff-e1a464bcccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:33.132643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d4817-775d-4bb2-b8ff-e1a464bcccd1 None admin] PATH_INFO: `/v3/users/5b4524f7f2f6410e983dadf2d798c568` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:33.133830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d4817-775d-4bb2-b8ff-e1a464bcccd1 None admin] RBAC: Authorizing `identity:get_user(user_id=5b4524f7f2f6410e983dadf2d798c568)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:33.135311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d4817-775d-4bb2-b8ff-e1a464bcccd1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:33.137169 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 41/85] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:32 2022] GET /identity/v3/users/5b4524f7f2f6410e983dadf2d798c568 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:33.137923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d4817-775d-4bb2-b8ff-e1a464bcccd1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:33.138716 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6512a8c-ec57-40c1-927e-b0a87165104d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:33.140358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6512a8c-ec57-40c1-927e-b0a87165104d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:33.236282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d4817-775d-4bb2-b8ff-e1a464bcccd1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:33.236748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 45/86] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:33.237480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6512a8c-ec57-40c1-927e-b0a87165104d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:33.274167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21e2b5-f427-4265-b44e-8dfe3b0cfa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:33.274789 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21e2b5-f427-4265-b44e-8dfe3b0cfa70 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:33.275103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21e2b5-f427-4265-b44e-8dfe3b0cfa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:33.275399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21e2b5-f427-4265-b44e-8dfe3b0cfa70 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:33.276077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21e2b5-f427-4265-b44e-8dfe3b0cfa70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:33.277529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21e2b5-f427-4265-b44e-8dfe3b0cfa70 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:33.291603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:33.303586 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 42/87] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:23:33 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:23:33.304396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21e2b5-f427-4265-b44e-8dfe3b0cfa70 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:33.352905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6512a8c-ec57-40c1-927e-b0a87165104d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:33.365490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:33.398647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e59d28-aa21-4ea2-a388-f1d0d1b5b163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:33.399622 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e59d28-aa21-4ea2-a388-f1d0d1b5b163 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:33.399908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e59d28-aa21-4ea2-a388-f1d0d1b5b163 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:33.400187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e59d28-aa21-4ea2-a388-f1d0d1b5b163 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:33.400997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e59d28-aa21-4ea2-a388-f1d0d1b5b163 None admin] RBAC: Adding query filter params (role.id=d8e3b1d89cfa4e0fb43b360a31b56605, scope.project.id=232b566033974d52b87f5e44eb52f5a2, user.id=5b4524f7f2f6410e983dadf2d798c568) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:33.401316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e59d28-aa21-4ea2-a388-f1d0d1b5b163 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:33.402875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e59d28-aa21-4ea2-a388-f1d0d1b5b163 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:33.423102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 46/88] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:33 2022] GET /identity/v3/role_assignments?user.id=5b4524f7f2f6410e983dadf2d798c568&scope.project.id=232b566033974d52b87f5e44eb52f5a2&role.id=d8e3b1d89cfa4e0fb43b360a31b56605 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:33.423102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e59d28-aa21-4ea2-a388-f1d0d1b5b163 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:33.439475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21e2b5-f427-4265-b44e-8dfe3b0cfa70 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:33.482470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bda1e-cce2-4d9f-9d67-a583630cabd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:33.483470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bda1e-cce2-4d9f-9d67-a583630cabd0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:33.483768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bda1e-cce2-4d9f-9d67-a583630cabd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:33.484064 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bda1e-cce2-4d9f-9d67-a583630cabd0 None admin] PATH_INFO: `/v3/users/5b4524f7f2f6410e983dadf2d798c568` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:33.485602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4bda1e-cce2-4d9f-9d67-a583630cabd0 None admin] RBAC: Authorizing `identity:get_user(user_id=5b4524f7f2f6410e983dadf2d798c568)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:33.487235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4bda1e-cce2-4d9f-9d67-a583630cabd0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:33.489506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 43/89] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:33 2022] GET /identity/v3/users/5b4524f7f2f6410e983dadf2d798c568 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:33.490357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bda1e-cce2-4d9f-9d67-a583630cabd0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:33.541063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e59d28-aa21-4ea2-a388-f1d0d1b5b163 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:33.563871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:33.586386 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebeaa5-576e-4be7-b74b-9b22c10ae5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:33.587016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebeaa5-576e-4be7-b74b-9b22c10ae5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:33.587337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebeaa5-576e-4be7-b74b-9b22c10ae5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:33.587630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebeaa5-576e-4be7-b74b-9b22c10ae5b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:33.628531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bda1e-cce2-4d9f-9d67-a583630cabd0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:33.649334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:33.674600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d1d70-b6e2-4772-8165-768ab6a6b7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:33.675364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ebeaa5-576e-4be7-b74b-9b22c10ae5b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:33.675755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d1d70-b6e2-4772-8165-768ab6a6b7ec None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:33.676181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d1d70-b6e2-4772-8165-768ab6a6b7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:33.676613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d1d70-b6e2-4772-8165-768ab6a6b7ec None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:33.676854 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ebeaa5-576e-4be7-b74b-9b22c10ae5b6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:33.678088 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d1d70-b6e2-4772-8165-768ab6a6b7ec None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:33.680767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d1d70-b6e2-4772-8165-768ab6a6b7ec None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:33.682742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 44/90] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:33 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:33.684160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d1d70-b6e2-4772-8165-768ab6a6b7ec None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:33.794877 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 47/91] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:33.796068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebeaa5-576e-4be7-b74b-9b22c10ae5b6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:33.816940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d1d70-b6e2-4772-8165-768ab6a6b7ec None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:33.847445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:33.862543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb621fe-cfa3-4683-8bfb-ab876f340cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:33.863565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb621fe-cfa3-4683-8bfb-ab876f340cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:33.864267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb621fe-cfa3-4683-8bfb-ab876f340cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:33.864958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb621fe-cfa3-4683-8bfb-ab876f340cc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:33.905322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebeaa5-576e-4be7-b74b-9b22c10ae5b6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:33.932960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:33.946219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18db311-1fdf-4ec1-941a-ad561a1fde41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:33.951992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18db311-1fdf-4ec1-941a-ad561a1fde41 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:33.952664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18db311-1fdf-4ec1-941a-ad561a1fde41 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:33.953197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18db311-1fdf-4ec1-941a-ad561a1fde41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:33.954262 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18db311-1fdf-4ec1-941a-ad561a1fde41 None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=232b566033974d52b87f5e44eb52f5a2, user.id=aae3d76a87c24d97b28248f570423938) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:33.954732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18db311-1fdf-4ec1-941a-ad561a1fde41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:33.955294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb621fe-cfa3-4683-8bfb-ab876f340cc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:33.963090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18db311-1fdf-4ec1-941a-ad561a1fde41 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:33.963241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb621fe-cfa3-4683-8bfb-ab876f340cc0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:33.973465 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 48/92] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:33 2022] GET /identity/v3/role_assignments?user.id=aae3d76a87c24d97b28248f570423938&scope.project.id=232b566033974d52b87f5e44eb52f5a2&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:33.974746 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18db311-1fdf-4ec1-941a-ad561a1fde41 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:34.103623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:34.116796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18db311-1fdf-4ec1-941a-ad561a1fde41 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:34.186198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cbace-8131-4551-965c-f885424b99e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:34.186949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cbace-8131-4551-965c-f885424b99e2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:34.187263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cbace-8131-4551-965c-f885424b99e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:34.187546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cbace-8131-4551-965c-f885424b99e2 None admin] PATH_INFO: `/v3/projects/ad0169ffda62481b93f959e15f033d13` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:34.189989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cbace-8131-4551-965c-f885424b99e2 None admin] RBAC: Authorizing `identity:get_project(project_id=ad0169ffda62481b93f959e15f033d13)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:34.191493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cbace-8131-4551-965c-f885424b99e2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:34.193186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 49/93] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:33 2022] GET /identity/v3/projects/ad0169ffda62481b93f959e15f033d13 => generated 312 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:34.203339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cbace-8131-4551-965c-f885424b99e2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:34.212380 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 45/94] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:34.218282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb621fe-cfa3-4683-8bfb-ab876f340cc0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:34.379262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:34.419003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb621fe-cfa3-4683-8bfb-ab876f340cc0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:34.437740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cbace-8131-4551-965c-f885424b99e2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:34.494534 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb2963-ff81-4d91-be45-a65b78a29e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:34.494732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74193ac3-73dc-4a10-93f0-799f9eaabf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:34.495185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74193ac3-73dc-4a10-93f0-799f9eaabf51 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:34.495287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb2963-ff81-4d91-be45-a65b78a29e12 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:34.495470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74193ac3-73dc-4a10-93f0-799f9eaabf51 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:34.495565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb2963-ff81-4d91-be45-a65b78a29e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:34.495748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74193ac3-73dc-4a10-93f0-799f9eaabf51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:34.495858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb2963-ff81-4d91-be45-a65b78a29e12 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:34.496554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74193ac3-73dc-4a10-93f0-799f9eaabf51 None admin] RBAC: Adding query filter params (role.id=d8e3b1d89cfa4e0fb43b360a31b56605, scope.project.id=ad0169ffda62481b93f959e15f033d13, user.id=5b4524f7f2f6410e983dadf2d798c568) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:34.496869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74193ac3-73dc-4a10-93f0-799f9eaabf51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:34.498561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74193ac3-73dc-4a10-93f0-799f9eaabf51 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:34.499794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb2963-ff81-4d91-be45-a65b78a29e12 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:34.501245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb2963-ff81-4d91-be45-a65b78a29e12 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:34.503270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 46/95] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:34 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:34.504591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 50/96] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:34 2022] GET /identity/v3/role_assignments?user.id=5b4524f7f2f6410e983dadf2d798c568&scope.project.id=ad0169ffda62481b93f959e15f033d13&role.id=d8e3b1d89cfa4e0fb43b360a31b56605 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:34.509354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb2963-ff81-4d91-be45-a65b78a29e12 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:34.650692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb2963-ff81-4d91-be45-a65b78a29e12 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:34.689256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946e43f-a27e-4722-8e69-17276dc53e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:34.689907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946e43f-a27e-4722-8e69-17276dc53e71 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:34.690196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946e43f-a27e-4722-8e69-17276dc53e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:34.690480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946e43f-a27e-4722-8e69-17276dc53e71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:34.691330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946e43f-a27e-4722-8e69-17276dc53e71 None admin] RBAC: Adding query filter params (role.id=0a56be0718ea495f8a241ae2447069cf, scope.project.id=232b566033974d52b87f5e44eb52f5a2, user.id=5b4524f7f2f6410e983dadf2d798c568) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:34.691650 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946e43f-a27e-4722-8e69-17276dc53e71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:34.693117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946e43f-a27e-4722-8e69-17276dc53e71 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:34.702012 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 47/97] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:34 2022] GET /identity/v3/role_assignments?user.id=5b4524f7f2f6410e983dadf2d798c568&scope.project.id=232b566033974d52b87f5e44eb52f5a2&role.id=0a56be0718ea495f8a241ae2447069cf => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:36.200363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4501fbe-0754-4ff4-9761-62193c6defab None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:36.200924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4501fbe-0754-4ff4-9761-62193c6defab None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:36.201041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4501fbe-0754-4ff4-9761-62193c6defab None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:36.202473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 51/98] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:36.217297 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79d4b-5390-4203-b868-78fc8d4bfc0a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:36.217898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79d4b-5390-4203-b868-78fc8d4bfc0a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:36.217898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79d4b-5390-4203-b868-78fc8d4bfc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:36.248698 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a7d79d4b-5390-4203-b868-78fc8d4bfc0a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:36.249931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7d79d4b-5390-4203-b868-78fc8d4bfc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:36.262410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:36.314407 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115be4f-95e1-4784-a525-fd209f95fe13 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:36.314763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115be4f-95e1-4784-a525-fd209f95fe13 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:36.314966 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115be4f-95e1-4784-a525-fd209f95fe13 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:36.316342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 52/99] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:36.321441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde8ab6-f0c5-4681-8938-a7b019dc5d35 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:36.321726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde8ab6-f0c5-4681-8938-a7b019dc5d35 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:36.322004 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde8ab6-f0c5-4681-8938-a7b019dc5d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:36.350019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bdde8ab6-f0c5-4681-8938-a7b019dc5d35 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:36.351245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bdde8ab6-f0c5-4681-8938-a7b019dc5d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:36.355618 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 48/100] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:36.364123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:36.391425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d79d4b-5390-4203-b868-78fc8d4bfc0a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:36.469825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 53/101] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:36.514400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde8ab6-f0c5-4681-8938-a7b019dc5d35 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:36.713661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde8ab6-f0c5-4681-8938-a7b019dc5d35 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:36.763037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dca488-d637-4d3e-a1aa-cb885068822b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:36.764353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dca488-d637-4d3e-a1aa-cb885068822b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:36.764872 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dca488-d637-4d3e-a1aa-cb885068822b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:36.765349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dca488-d637-4d3e-a1aa-cb885068822b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:36.792855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dca488-d637-4d3e-a1aa-cb885068822b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-e9dca488-d637-4d3e-a1aa-cb885068822b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:36.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:36.809189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:36.809931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:36.809931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:36.809931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:36.809931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:36.809931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:36.809931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:36.809931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:36.809931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:36.809931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:36.809931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:36.809931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:36.809931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:36.813273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 54/102] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:36.814885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e393880-8e43-41be-9585-3e73206b0d5c None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:36.815436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e393880-8e43-41be-9585-3e73206b0d5c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:36.815961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e393880-8e43-41be-9585-3e73206b0d5c None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:36.817497 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 55/103] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:36.821460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d79d4b-5390-4203-b868-78fc8d4bfc0a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:36.822730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e393880-8e43-41be-9585-3e73206b0d5c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:36.830181 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:36.911185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26380d7-e4b1-4df6-81bd-e7684d116fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:36.911852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26380d7-e4b1-4df6-81bd-e7684d116fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:36.912138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26380d7-e4b1-4df6-81bd-e7684d116fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:36.912421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26380d7-e4b1-4df6-81bd-e7684d116fb6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:36.935751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26380d7-e4b1-4df6-81bd-e7684d116fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d26380d7-e4b1-4df6-81bd-e7684d116fb6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:36.941582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:36.942492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:36.943290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:36.943290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:36.943290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:36.943290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:36.943290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:36.943290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:36.943290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:36.943290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:36.943290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:36.943290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:36.943290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:23:36.943290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:36.946706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 49/104] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 556 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:23:36.949641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0220732-db0b-4672-9016-94ceb3d5a5a5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:36.950160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0220732-db0b-4672-9016-94ceb3d5a5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:36.950642 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0220732-db0b-4672-9016-94ceb3d5a5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:36.972785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:36.979254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e393880-8e43-41be-9585-3e73206b0d5c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:36.989006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e0220732-db0b-4672-9016-94ceb3d5a5a5 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:36.990362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e0220732-db0b-4672-9016-94ceb3d5a5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:37.060045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98314a-fba9-4a6a-a8b4-c36a5497447a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:37.060724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98314a-fba9-4a6a-a8b4-c36a5497447a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.061011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98314a-fba9-4a6a-a8b4-c36a5497447a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.061290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98314a-fba9-4a6a-a8b4-c36a5497447a None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:37.062042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98314a-fba9-4a6a-a8b4-c36a5497447a None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:37.062354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98314a-fba9-4a6a-a8b4-c36a5497447a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:37.064132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98314a-fba9-4a6a-a8b4-c36a5497447a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:37.074342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 56/105] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:37.075951 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc72e2-7989-4d2e-a551-d4744c566c69 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.076234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc72e2-7989-4d2e-a551-d4744c566c69 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.076514 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc72e2-7989-4d2e-a551-d4744c566c69 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:37.079534 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 57/106] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:37.080705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc72e2-7989-4d2e-a551-d4744c566c69 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:37.151972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 50/107] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:37.153149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0220732-db0b-4672-9016-94ceb3d5a5a5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:37.184744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:37.225432 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc72e2-7989-4d2e-a551-d4744c566c69 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:37.262830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a42a3-d5ac-4476-834c-a0c089c2d232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:37.264097 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a42a3-d5ac-4476-834c-a0c089c2d232 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.264384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a42a3-d5ac-4476-834c-a0c089c2d232 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.264717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a42a3-d5ac-4476-834c-a0c089c2d232 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:37.265434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a42a3-d5ac-4476-834c-a0c089c2d232 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:37.265793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a42a3-d5ac-4476-834c-a0c089c2d232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:37.267589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a42a3-d5ac-4476-834c-a0c089c2d232 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:37.276610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 58/108] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:37.277819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380326b0-1922-4ebf-97b9-d8d2f1ca908a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.278109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380326b0-1922-4ebf-97b9-d8d2f1ca908a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.278392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380326b0-1922-4ebf-97b9-d8d2f1ca908a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:37.291537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0220732-db0b-4672-9016-94ceb3d5a5a5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:37.296881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-380326b0-1922-4ebf-97b9-d8d2f1ca908a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:37.298067 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-380326b0-1922-4ebf-97b9-d8d2f1ca908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:37.334131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee472ead-0008-4052-8fad-13363f30ab59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:37.335054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee472ead-0008-4052-8fad-13363f30ab59 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.335385 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee472ead-0008-4052-8fad-13363f30ab59 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.335675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee472ead-0008-4052-8fad-13363f30ab59 None admin] PATH_INFO: `/v3/users/5b4524f7f2f6410e983dadf2d798c568` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:37.336885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee472ead-0008-4052-8fad-13363f30ab59 None admin] RBAC: Authorizing `identity:get_user(user_id=5b4524f7f2f6410e983dadf2d798c568)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:37.338357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee472ead-0008-4052-8fad-13363f30ab59 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:37.340038 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 51/109] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:37 2022] GET /identity/v3/users/5b4524f7f2f6410e983dadf2d798c568 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:37.340806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee472ead-0008-4052-8fad-13363f30ab59 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:37.344468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:37.374453 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 59/110] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:37.375720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8fde6-91b2-43b3-8c21-83a40c993527 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.376020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8fde6-91b2-43b3-8c21-83a40c993527 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.376357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8fde6-91b2-43b3-8c21-83a40c993527 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:37.377719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 60/111] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:37.378345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8fde6-91b2-43b3-8c21-83a40c993527 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:37.445910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:37.466539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee472ead-0008-4052-8fad-13363f30ab59 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:37.492236 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8fde6-91b2-43b3-8c21-83a40c993527 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:37.501312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926cd99-6a7e-493f-badd-31c87c66d7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:37.501998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926cd99-6a7e-493f-badd-31c87c66d7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.502287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926cd99-6a7e-493f-badd-31c87c66d7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.502560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926cd99-6a7e-493f-badd-31c87c66d7c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:37.503737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926cd99-6a7e-493f-badd-31c87c66d7c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:37.505193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926cd99-6a7e-493f-badd-31c87c66d7c2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:37.511643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 52/112] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:37.511643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926cd99-6a7e-493f-badd-31c87c66d7c2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:37.535825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edbbb6-5f9f-40a9-b5b7-546f8c83a3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:37.536478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edbbb6-5f9f-40a9-b5b7-546f8c83a3de None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.536773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edbbb6-5f9f-40a9-b5b7-546f8c83a3de None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.537061 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edbbb6-5f9f-40a9-b5b7-546f8c83a3de None admin] PATH_INFO: `/v3/users/a59960e5047d40518f4471b4de5aae65` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:37.551689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90edbbb6-5f9f-40a9-b5b7-546f8c83a3de None admin] RBAC: Authorizing `identity:get_user(user_id=a59960e5047d40518f4471b4de5aae65)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:37.553177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90edbbb6-5f9f-40a9-b5b7-546f8c83a3de None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:37.554863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 61/113] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:37 2022] GET /identity/v3/users/a59960e5047d40518f4471b4de5aae65 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:37.556047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac434323-fa3a-4032-b6eb-0fe14ac9f0a5 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.556343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac434323-fa3a-4032-b6eb-0fe14ac9f0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.556628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac434323-fa3a-4032-b6eb-0fe14ac9f0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:37.574185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ac434323-fa3a-4032-b6eb-0fe14ac9f0a5 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:37.575407 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ac434323-fa3a-4032-b6eb-0fe14ac9f0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:37.590772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:37.625128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926cd99-6a7e-493f-badd-31c87c66d7c2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:37.648086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 62/114] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:37.648708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac434323-fa3a-4032-b6eb-0fe14ac9f0a5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:37.675163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74bbc3-d431-48ad-8328-895ec01614a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:37.676903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74bbc3-d431-48ad-8328-895ec01614a1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.677396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74bbc3-d431-48ad-8328-895ec01614a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.677866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74bbc3-d431-48ad-8328-895ec01614a1 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:37.679386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74bbc3-d431-48ad-8328-895ec01614a1 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:37.681053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74bbc3-d431-48ad-8328-895ec01614a1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:37.683190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 53/115] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:37 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:37.684604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74bbc3-d431-48ad-8328-895ec01614a1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:37.704517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:37.760626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac434323-fa3a-4032-b6eb-0fe14ac9f0a5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:37.794076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74bbc3-d431-48ad-8328-895ec01614a1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:37.798959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963336c-b529-4901-a726-7b574a1c078d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:37.799652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963336c-b529-4901-a726-7b574a1c078d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.799943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963336c-b529-4901-a726-7b574a1c078d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.800230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963336c-b529-4901-a726-7b574a1c078d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:37.811945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963336c-b529-4901-a726-7b574a1c078d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-5963336c-b529-4901-a726-7b574a1c078d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:37.816883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:37.817920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 63/116] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:37.818861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963336c-b529-4901-a726-7b574a1c078d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:37.839658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9e606-4190-4321-9b47-efceefeab06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:37.840294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9e606-4190-4321-9b47-efceefeab06b None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.840585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9e606-4190-4321-9b47-efceefeab06b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.840922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9e606-4190-4321-9b47-efceefeab06b None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:37.842093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9e606-4190-4321-9b47-efceefeab06b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:37.843596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9e606-4190-4321-9b47-efceefeab06b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:37.855171 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:37.871109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 54/117] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:23:37 2022] POST /identity/v3/projects => generated 302 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:23:37.879225 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9e606-4190-4321-9b47-efceefeab06b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:37.951262 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963336c-b529-4901-a726-7b574a1c078d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:37.964757 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:37.993922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9e606-4190-4321-9b47-efceefeab06b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:37.994543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ae14-66cb-411a-ac96-50ede2b0f27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:37.995208 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ae14-66cb-411a-ac96-50ede2b0f27c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:37.995495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ae14-66cb-411a-ac96-50ede2b0f27c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:37.995779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ae14-66cb-411a-ac96-50ede2b0f27c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:38.034532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7e561-b088-46c3-be84-f7aef9dbf233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:38.035600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7e561-b088-46c3-be84-f7aef9dbf233 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:38.036036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7e561-b088-46c3-be84-f7aef9dbf233 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:38.036540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7e561-b088-46c3-be84-f7aef9dbf233 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:38.054644 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7e561-b088-46c3-be84-f7aef9dbf233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-70d7e561-b088-46c3-be84-f7aef9dbf233 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:38.059662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:38.060779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:38.061548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 55/118] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:38.062177 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7e561-b088-46c3-be84-f7aef9dbf233 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:38.078688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0ae14-66cb-411a-ac96-50ede2b0f27c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:38.080601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0ae14-66cb-411a-ac96-50ede2b0f27c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:38.128127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:38.183638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7e561-b088-46c3-be84-f7aef9dbf233 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:38.187151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 64/119] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:38.188177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ae14-66cb-411a-ac96-50ede2b0f27c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:38.196631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:38.225292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5af331-2a41-4d64-8275-9ec638b80195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:38.225971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5af331-2a41-4d64-8275-9ec638b80195 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:38.226261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5af331-2a41-4d64-8275-9ec638b80195 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:38.226545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5af331-2a41-4d64-8275-9ec638b80195 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2/users/5b4524f7f2f6410e983dadf2d798c568/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:38.228562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5af331-2a41-4d64-8275-9ec638b80195 None admin] RBAC: Authorizing `identity:create_grant(project_id=232b566033974d52b87f5e44eb52f5a2, user_id=5b4524f7f2f6410e983dadf2d798c568, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:38.230040 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5af331-2a41-4d64-8275-9ec638b80195 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:38.245750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 56/120] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:23:38 2022] PUT /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2/users/5b4524f7f2f6410e983dadf2d798c568/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:23:38.246745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6bab3-442d-4662-917c-5bc13123f33b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:38.247041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6bab3-442d-4662-917c-5bc13123f33b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:38.247358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6bab3-442d-4662-917c-5bc13123f33b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:38.249036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 57/121] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:38.249648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6bab3-442d-4662-917c-5bc13123f33b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:38.320239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ae14-66cb-411a-ac96-50ede2b0f27c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:38.368310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233bbeb-1161-4fad-9b36-5f44f26a71d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:38.368965 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233bbeb-1161-4fad-9b36-5f44f26a71d5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:38.369255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233bbeb-1161-4fad-9b36-5f44f26a71d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:38.369535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233bbeb-1161-4fad-9b36-5f44f26a71d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:38.370271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233bbeb-1161-4fad-9b36-5f44f26a71d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:38.370580 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233bbeb-1161-4fad-9b36-5f44f26a71d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:38.372069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233bbeb-1161-4fad-9b36-5f44f26a71d5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:38.379123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6bab3-442d-4662-917c-5bc13123f33b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:38.383790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 65/122] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:38.384776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233bbeb-1161-4fad-9b36-5f44f26a71d5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:38.400059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:38.431939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987d942-a11a-4379-8358-6216676f3e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:38.432579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987d942-a11a-4379-8358-6216676f3e9f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:38.432868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987d942-a11a-4379-8358-6216676f3e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:38.433149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987d942-a11a-4379-8358-6216676f3e9f None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:38.433885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987d942-a11a-4379-8358-6216676f3e9f None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:38.434198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987d942-a11a-4379-8358-6216676f3e9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:38.436639 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987d942-a11a-4379-8358-6216676f3e9f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:38.448225 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 58/123] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:38.449333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c079c2-c405-45ed-bbda-a5ade0589340 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:38.449601 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c079c2-c405-45ed-bbda-a5ade0589340 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:38.449886 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c079c2-c405-45ed-bbda-a5ade0589340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:38.469132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:38.473782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-17c079c2-c405-45ed-bbda-a5ade0589340 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:38.477352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-17c079c2-c405-45ed-bbda-a5ade0589340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:38.510573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233bbeb-1161-4fad-9b36-5f44f26a71d5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:38.546530 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 59/124] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:38.547377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db6555-1234-4d8b-9a19-cf7041265086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:38.547946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c079c2-c405-45ed-bbda-a5ade0589340 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:38.548100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db6555-1234-4d8b-9a19-cf7041265086 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:38.548396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db6555-1234-4d8b-9a19-cf7041265086 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:38.548679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db6555-1234-4d8b-9a19-cf7041265086 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:38.549869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db6555-1234-4d8b-9a19-cf7041265086 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:38.551547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db6555-1234-4d8b-9a19-cf7041265086 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:38.553490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 66/125] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:38 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:38.554330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db6555-1234-4d8b-9a19-cf7041265086 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:38.660942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c079c2-c405-45ed-bbda-a5ade0589340 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:38.663352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:38.682973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db6555-1234-4d8b-9a19-cf7041265086 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:38.697864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeea6f2-0b73-4968-acfa-34c51a3145b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:38.698630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeea6f2-0b73-4968-acfa-34c51a3145b6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:38.698917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeea6f2-0b73-4968-acfa-34c51a3145b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:38.699226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeea6f2-0b73-4968-acfa-34c51a3145b6 None admin] PATH_INFO: `/v3/users/aae3d76a87c24d97b28248f570423938` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:38.700404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeea6f2-0b73-4968-acfa-34c51a3145b6 None admin] RBAC: Authorizing `identity:get_user(user_id=aae3d76a87c24d97b28248f570423938)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:38.703017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeea6f2-0b73-4968-acfa-34c51a3145b6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:38.705146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 60/126] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:38 2022] GET /identity/v3/users/aae3d76a87c24d97b28248f570423938 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:38.706027 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeea6f2-0b73-4968-acfa-34c51a3145b6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:38.727271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810a310-062d-4fe4-add7-b61e91f88655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:38.727979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810a310-062d-4fe4-add7-b61e91f88655 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:38.728270 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810a310-062d-4fe4-add7-b61e91f88655 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:38.728556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810a310-062d-4fe4-add7-b61e91f88655 None admin] PATH_INFO: `/v3/users/5b4524f7f2f6410e983dadf2d798c568` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:38.729896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810a310-062d-4fe4-add7-b61e91f88655 None admin] RBAC: Authorizing `identity:get_user(user_id=5b4524f7f2f6410e983dadf2d798c568)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:38.731554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810a310-062d-4fe4-add7-b61e91f88655 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:38.733566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 67/127] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:38 2022] GET /identity/v3/users/5b4524f7f2f6410e983dadf2d798c568 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:38.734335 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810a310-062d-4fe4-add7-b61e91f88655 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:38.739359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:38.818257 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeea6f2-0b73-4968-acfa-34c51a3145b6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:38.850058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810a310-062d-4fe4-add7-b61e91f88655 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:38.854005 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efe159-71ca-42a0-8052-fcad37d17faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:38.854816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efe159-71ca-42a0-8052-fcad37d17faa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:38.855267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efe159-71ca-42a0-8052-fcad37d17faa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:38.855685 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efe159-71ca-42a0-8052-fcad37d17faa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:38.856603 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86efe159-71ca-42a0-8052-fcad37d17faa None admin] RBAC: Adding query filter params (role.id=84c485f3441a4653b579e9ab39d98a95, scope.project.id=232b566033974d52b87f5e44eb52f5a2, user.id=a59960e5047d40518f4471b4de5aae65) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:38.857624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86efe159-71ca-42a0-8052-fcad37d17faa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:38.859313 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86efe159-71ca-42a0-8052-fcad37d17faa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:38.865415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 61/128] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:38 2022] GET /identity/v3/role_assignments?user.id=a59960e5047d40518f4471b4de5aae65&scope.project.id=232b566033974d52b87f5e44eb52f5a2&role.id=84c485f3441a4653b579e9ab39d98a95 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:38.866354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efe159-71ca-42a0-8052-fcad37d17faa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:38.907513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:38.912815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231011e-a2e6-4426-ad43-f5a314ffc1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:38.913711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231011e-a2e6-4426-ad43-f5a314ffc1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:38.913711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231011e-a2e6-4426-ad43-f5a314ffc1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:38.913937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231011e-a2e6-4426-ad43-f5a314ffc1d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:38.935872 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231011e-a2e6-4426-ad43-f5a314ffc1d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-b231011e-a2e6-4426-ad43-f5a314ffc1d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:38.950144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:38.951416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:38.952195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:38.952195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:38.952195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:38.952195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:38.952195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:38.952195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:38.952195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:38.952195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:38.952195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:38.952195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:38.952195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:23:38.952195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:38.953646 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 68/129] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:23:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:38.954288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231011e-a2e6-4426-ad43-f5a314ffc1d7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:38.994895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efe159-71ca-42a0-8052-fcad37d17faa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:39.034433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771fdec-c745-4156-8bc6-ed5f34e792d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:39.035424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771fdec-c745-4156-8bc6-ed5f34e792d3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:39.035907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771fdec-c745-4156-8bc6-ed5f34e792d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:39.036329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771fdec-c745-4156-8bc6-ed5f34e792d3 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:39.037730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771fdec-c745-4156-8bc6-ed5f34e792d3 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:39.039374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771fdec-c745-4156-8bc6-ed5f34e792d3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:39.041617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 62/130] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:38 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:39.042721 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771fdec-c745-4156-8bc6-ed5f34e792d3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:39.061447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:39.132864 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231011e-a2e6-4426-ad43-f5a314ffc1d7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:39.159557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771fdec-c745-4156-8bc6-ed5f34e792d3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:39.162369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:39.199533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b139f49-c133-4c22-9108-2ba0d526d576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:39.200461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b139f49-c133-4c22-9108-2ba0d526d576 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:39.200957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b139f49-c133-4c22-9108-2ba0d526d576 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:39.201375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b139f49-c133-4c22-9108-2ba0d526d576 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:39.202360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b139f49-c133-4c22-9108-2ba0d526d576 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:39.202810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b139f49-c133-4c22-9108-2ba0d526d576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:39.203846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a3db3-9bfc-495e-ad52-b84af22547a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:39.204517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a3db3-9bfc-495e-ad52-b84af22547a3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:39.204731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b139f49-c133-4c22-9108-2ba0d526d576 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:39.204836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a3db3-9bfc-495e-ad52-b84af22547a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:39.205080 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a3db3-9bfc-495e-ad52-b84af22547a3 None admin] PATH_INFO: `/v3/projects/ad0169ffda62481b93f959e15f033d13` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:39.206353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a3db3-9bfc-495e-ad52-b84af22547a3 None admin] RBAC: Authorizing `identity:get_project(project_id=ad0169ffda62481b93f959e15f033d13)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:39.208140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a3db3-9bfc-495e-ad52-b84af22547a3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:39.209585 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 69/131] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:38 2022] GET /identity/v3/projects/ad0169ffda62481b93f959e15f033d13 => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:39.210331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a3db3-9bfc-495e-ad52-b84af22547a3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:39.220397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 63/132] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:23:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:39.221517 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b139f49-c133-4c22-9108-2ba0d526d576 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:39.330264 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b139f49-c133-4c22-9108-2ba0d526d576 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:39.368821 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3ff61-b681-40ee-989d-5c871b1f2d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:39.369727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3ff61-b681-40ee-989d-5c871b1f2d8e None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:39.370353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3ff61-b681-40ee-989d-5c871b1f2d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:39.371399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3ff61-b681-40ee-989d-5c871b1f2d8e None admin] PATH_INFO: `/v3/projects/ad0169ffda62481b93f959e15f033d13/users/5b4524f7f2f6410e983dadf2d798c568/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:39.374374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c3ff61-b681-40ee-989d-5c871b1f2d8e None admin] RBAC: Authorizing `identity:create_grant(project_id=ad0169ffda62481b93f959e15f033d13, user_id=5b4524f7f2f6410e983dadf2d798c568, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:39.377281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c3ff61-b681-40ee-989d-5c871b1f2d8e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:39.397131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 64/133] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:23:39 2022] PUT /identity/v3/projects/ad0169ffda62481b93f959e15f033d13/users/5b4524f7f2f6410e983dadf2d798c568/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:23:39.407103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3ff61-b681-40ee-989d-5c871b1f2d8e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:39.428451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a3db3-9bfc-495e-ad52-b84af22547a3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:39.477932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:39.506207 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:39.515938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7dace-04f5-46f6-a219-5a49ab3e0772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:39.516650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7dace-04f5-46f6-a219-5a49ab3e0772 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:39.516936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7dace-04f5-46f6-a219-5a49ab3e0772 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:39.517213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7dace-04f5-46f6-a219-5a49ab3e0772 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2/users/aae3d76a87c24d97b28248f570423938/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:39.521375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7dace-04f5-46f6-a219-5a49ab3e0772 None admin] RBAC: Authorizing `identity:create_grant(project_id=232b566033974d52b87f5e44eb52f5a2, user_id=aae3d76a87c24d97b28248f570423938, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:39.522885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7dace-04f5-46f6-a219-5a49ab3e0772 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:39.553292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 70/134] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:23:39 2022] PUT /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2/users/aae3d76a87c24d97b28248f570423938/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:23:39.601933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3ff61-b681-40ee-989d-5c871b1f2d8e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:39.642535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b34ab9-d1b1-44f4-91c1-bc09141ff5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:39.643226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b34ab9-d1b1-44f4-91c1-bc09141ff5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:39.643520 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b34ab9-d1b1-44f4-91c1-bc09141ff5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:39.643802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b34ab9-d1b1-44f4-91c1-bc09141ff5a8 None admin] PATH_INFO: `/v3/users/5b4524f7f2f6410e983dadf2d798c568` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:39.645039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b34ab9-d1b1-44f4-91c1-bc09141ff5a8 None admin] RBAC: Authorizing `identity:get_user(user_id=5b4524f7f2f6410e983dadf2d798c568)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:39.646559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b34ab9-d1b1-44f4-91c1-bc09141ff5a8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:39.648281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 65/135] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:39 2022] GET /identity/v3/users/5b4524f7f2f6410e983dadf2d798c568 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:39.654722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7dace-04f5-46f6-a219-5a49ab3e0772 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:39.782215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7dace-04f5-46f6-a219-5a49ab3e0772 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:39.819510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022a0b4-e395-40de-9743-b81eff9ba867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:39.820170 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a0b4-e395-40de-9743-b81eff9ba867 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:39.820458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a0b4-e395-40de-9743-b81eff9ba867 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:39.820736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a0b4-e395-40de-9743-b81eff9ba867 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:39.821950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022a0b4-e395-40de-9743-b81eff9ba867 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:39.823440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022a0b4-e395-40de-9743-b81eff9ba867 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:39.825106 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 71/136] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:39 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:39.831506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b34ab9-d1b1-44f4-91c1-bc09141ff5a8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:39.931160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:39.941484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b34ab9-d1b1-44f4-91c1-bc09141ff5a8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:39.986174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c58a1d-30e8-4f11-a66d-f56dfb0fc6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:39.986950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c58a1d-30e8-4f11-a66d-f56dfb0fc6e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:39.987324 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c58a1d-30e8-4f11-a66d-f56dfb0fc6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:39.987669 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c58a1d-30e8-4f11-a66d-f56dfb0fc6e0 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2/users/5b4524f7f2f6410e983dadf2d798c568/roles/0a56be0718ea495f8a241ae2447069cf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:39.992855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c58a1d-30e8-4f11-a66d-f56dfb0fc6e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=232b566033974d52b87f5e44eb52f5a2, user_id=5b4524f7f2f6410e983dadf2d798c568, role_id=0a56be0718ea495f8a241ae2447069cf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:39.994381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c58a1d-30e8-4f11-a66d-f56dfb0fc6e0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:40.016310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 66/137] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:23:39 2022] PUT /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2/users/5b4524f7f2f6410e983dadf2d798c568/roles/0a56be0718ea495f8a241ae2447069cf => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:23:40.467657 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc39f97-b939-4d19-8f26-c6104c86d3d9 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:40.468186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc39f97-b939-4d19-8f26-c6104c86d3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:40.468490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc39f97-b939-4d19-8f26-c6104c86d3d9 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:40.470186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 72/138] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:40.476180 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6713f2-525c-4192-b22a-30ae3c73e40d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:40.476608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6713f2-525c-4192-b22a-30ae3c73e40d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:40.476823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6713f2-525c-4192-b22a-30ae3c73e40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:40.496135 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4d6713f2-525c-4192-b22a-30ae3c73e40d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:40.497416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4d6713f2-525c-4192-b22a-30ae3c73e40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:40.568790 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 67/139] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:40.592896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc39f97-b939-4d19-8f26-c6104c86d3d9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:40.658911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:40.703776 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c50b3-49da-4607-a5fb-a939ba89ddb3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:40.704782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c50b3-49da-4607-a5fb-a939ba89ddb3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:40.705554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c50b3-49da-4607-a5fb-a939ba89ddb3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:40.711789 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 68/140] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:23:40.718562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc39f97-b939-4d19-8f26-c6104c86d3d9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:40.719500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa754d0-00a8-4905-b94e-1589bc4b3a23 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:40.720202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa754d0-00a8-4905-b94e-1589bc4b3a23 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:40.720692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa754d0-00a8-4905-b94e-1589bc4b3a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:40.756032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7fa754d0-00a8-4905-b94e-1589bc4b3a23 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:40.757811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7fa754d0-00a8-4905-b94e-1589bc4b3a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:40.806392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e20b1-d8ac-482a-8b38-7aa76607e951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:40.807040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e20b1-d8ac-482a-8b38-7aa76607e951 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:40.807350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e20b1-d8ac-482a-8b38-7aa76607e951 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:40.807626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e20b1-d8ac-482a-8b38-7aa76607e951 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:40.810432 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e20b1-d8ac-482a-8b38-7aa76607e951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:40.819020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e20b1-d8ac-482a-8b38-7aa76607e951 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:40.823044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 73/141] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:40.831178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e20b1-d8ac-482a-8b38-7aa76607e951 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:40.952562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 69/142] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:40.992627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa754d0-00a8-4905-b94e-1589bc4b3a23 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:40.996933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e20b1-d8ac-482a-8b38-7aa76607e951 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:41.024580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:41.037628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:41.057068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39490282-4c5b-44ff-8a7e-6e28b33f7d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:41.057711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39490282-4c5b-44ff-8a7e-6e28b33f7d13 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:41.057993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39490282-4c5b-44ff-8a7e-6e28b33f7d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:41.058272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39490282-4c5b-44ff-8a7e-6e28b33f7d13 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:41.058967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39490282-4c5b-44ff-8a7e-6e28b33f7d13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:41.061691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39490282-4c5b-44ff-8a7e-6e28b33f7d13 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:41.105063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 74/143] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:23:40 2022] POST /identity/v3/users => generated 283 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:23:41.191755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa754d0-00a8-4905-b94e-1589bc4b3a23 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:41.234296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedab48-c41f-442f-acab-a4d2881c5a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:41.234946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedab48-c41f-442f-acab-a4d2881c5a4e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:41.235262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedab48-c41f-442f-acab-a4d2881c5a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:41.235540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedab48-c41f-442f-acab-a4d2881c5a4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:41.246403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aedab48-c41f-442f-acab-a4d2881c5a4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5aedab48-c41f-442f-acab-a4d2881c5a4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:41.252143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:41.253072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:41.253861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 70/144] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:23:41.259348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39490282-4c5b-44ff-8a7e-6e28b33f7d13 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:41.392514 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39490282-4c5b-44ff-8a7e-6e28b33f7d13 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:41.440669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7e17a-9e53-4e76-90d7-88d8c6af75b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:41.441302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7e17a-9e53-4e76-90d7-88d8c6af75b4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:41.441586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7e17a-9e53-4e76-90d7-88d8c6af75b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:41.441860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7e17a-9e53-4e76-90d7-88d8c6af75b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:41.442576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7e17a-9e53-4e76-90d7-88d8c6af75b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:41.442879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7e17a-9e53-4e76-90d7-88d8c6af75b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:41.444298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7e17a-9e53-4e76-90d7-88d8c6af75b4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:41.458188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 75/145] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:41.463934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedab48-c41f-442f-acab-a4d2881c5a4e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:41.542974 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:41.569090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedab48-c41f-442f-acab-a4d2881c5a4e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:41.605175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01c3eb-e4b1-4802-abc2-c0a3f7d98dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:41.605825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01c3eb-e4b1-4802-abc2-c0a3f7d98dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:41.606110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01c3eb-e4b1-4802-abc2-c0a3f7d98dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:41.606392 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01c3eb-e4b1-4802-abc2-c0a3f7d98dd2 None admin] PATH_INFO: `/v3/users/5b4524f7f2f6410e983dadf2d798c568` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:41.608473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01c3eb-e4b1-4802-abc2-c0a3f7d98dd2 None admin] RBAC: Authorizing `identity:get_user(user_id=5b4524f7f2f6410e983dadf2d798c568)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:41.610017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01c3eb-e4b1-4802-abc2-c0a3f7d98dd2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:41.611849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 71/146] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:41 2022] GET /identity/v3/users/5b4524f7f2f6410e983dadf2d798c568 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:41.620721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7e17a-9e53-4e76-90d7-88d8c6af75b4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:41.679464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:41.743197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7e17a-9e53-4e76-90d7-88d8c6af75b4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:41.785658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0faa11-b9ac-40b5-9fa4-8ce75e2ff82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:41.786337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0faa11-b9ac-40b5-9fa4-8ce75e2ff82f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:41.786619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0faa11-b9ac-40b5-9fa4-8ce75e2ff82f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:41.786937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0faa11-b9ac-40b5-9fa4-8ce75e2ff82f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:41.852229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ed4ba-ad22-4eed-b1cf-fe63861d138d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:41.852547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ed4ba-ad22-4eed-b1cf-fe63861d138d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:41.852714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ed4ba-ad22-4eed-b1cf-fe63861d138d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:41.854057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 72/147] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:23:41.859146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf148f0-d129-47c6-be5e-c458c79e8a11 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:41.859465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf148f0-d129-47c6-be5e-c458c79e8a11 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:41.859726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf148f0-d129-47c6-be5e-c458c79e8a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:41.883918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0faa11-b9ac-40b5-9fa4-8ce75e2ff82f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:41.885408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0faa11-b9ac-40b5-9fa4-8ce75e2ff82f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:41.890486 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5bf148f0-d129-47c6-be5e-c458c79e8a11 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:41.891720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5bf148f0-d129-47c6-be5e-c458c79e8a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:41.972466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:42.005510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 73/148] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:42.042900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf148f0-d129-47c6-be5e-c458c79e8a11 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:42.068109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 76/149] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:42.073694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0faa11-b9ac-40b5-9fa4-8ce75e2ff82f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:42.155116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:42.188088 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf148f0-d129-47c6-be5e-c458c79e8a11 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:42.245032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0faa11-b9ac-40b5-9fa4-8ce75e2ff82f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:42.280373 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316dac2-073c-4c3a-ba8e-01c0f4dcd3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:42.281416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316dac2-073c-4c3a-ba8e-01c0f4dcd3ad None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:42.281912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316dac2-073c-4c3a-ba8e-01c0f4dcd3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:42.282393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316dac2-073c-4c3a-ba8e-01c0f4dcd3ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:42.290817 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5e6c4-c142-49c1-9ebf-ead7f2d18fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:42.291818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5e6c4-c142-49c1-9ebf-ead7f2d18fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:42.292302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5e6c4-c142-49c1-9ebf-ead7f2d18fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:42.292799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5e6c4-c142-49c1-9ebf-ead7f2d18fe1 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:42.295027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5e6c4-c142-49c1-9ebf-ead7f2d18fe1 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:42.296784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5e6c4-c142-49c1-9ebf-ead7f2d18fe1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:42.298785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 77/150] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:42 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:42.304452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316dac2-073c-4c3a-ba8e-01c0f4dcd3ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:42.304972 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5e6c4-c142-49c1-9ebf-ead7f2d18fe1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b316dac2-073c-4c3a-ba8e-01c0f4dcd3ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:42.309420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:42.310820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:42.311627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:42.311627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:42.311627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:42.311627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:42.311627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:42.311627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:42.311627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:42.311627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:42.311627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:42.311627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:42.311627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:23:42.311627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:42.317316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 74/151] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:23:42.318409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316dac2-073c-4c3a-ba8e-01c0f4dcd3ad None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:42.341656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:42.442751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316dac2-073c-4c3a-ba8e-01c0f4dcd3ad None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:42.504429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:42.507213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5e6c4-c142-49c1-9ebf-ead7f2d18fe1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:42.521179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ddb6b-5432-4c94-9b0c-d4f22a1b426a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:42.521824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ddb6b-5432-4c94-9b0c-d4f22a1b426a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:42.522110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ddb6b-5432-4c94-9b0c-d4f22a1b426a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:42.522392 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ddb6b-5432-4c94-9b0c-d4f22a1b426a None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:42.523146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ddb6b-5432-4c94-9b0c-d4f22a1b426a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:42.523460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ddb6b-5432-4c94-9b0c-d4f22a1b426a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:42.524859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ddb6b-5432-4c94-9b0c-d4f22a1b426a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:42.542601 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 75/152] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:42.543921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb37b04-9078-4385-9814-dce5dbb34f04 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:42.544981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb37b04-9078-4385-9814-dce5dbb34f04 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:42.545052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb37b04-9078-4385-9814-dce5dbb34f04 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:42.546305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 76/153] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:42.548990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae5f3e-9030-4a45-bb49-96743a01f569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:42.549621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae5f3e-9030-4a45-bb49-96743a01f569 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:42.549905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae5f3e-9030-4a45-bb49-96743a01f569 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:42.550184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae5f3e-9030-4a45-bb49-96743a01f569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:42.550986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae5f3e-9030-4a45-bb49-96743a01f569 None admin] RBAC: Adding query filter params (role.id=d8e3b1d89cfa4e0fb43b360a31b56605, scope.project.id=232b566033974d52b87f5e44eb52f5a2, user.id=5b4524f7f2f6410e983dadf2d798c568) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:42.551326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae5f3e-9030-4a45-bb49-96743a01f569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:42.552800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae5f3e-9030-4a45-bb49-96743a01f569 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:42.563500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 78/154] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:42 2022] GET /identity/v3/role_assignments?user.id=5b4524f7f2f6410e983dadf2d798c568&scope.project.id=232b566033974d52b87f5e44eb52f5a2&role.id=d8e3b1d89cfa4e0fb43b360a31b56605 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:42.564615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a33408-768a-405e-b3a0-31553d3ccfe3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:42.564902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a33408-768a-405e-b3a0-31553d3ccfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:42.565185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a33408-768a-405e-b3a0-31553d3ccfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:42.565734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae5f3e-9030-4a45-bb49-96743a01f569 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:42.604125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-77a33408-768a-405e-b3a0-31553d3ccfe3 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:42.605365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-77a33408-768a-405e-b3a0-31553d3ccfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:42.664314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:42.690389 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae5f3e-9030-4a45-bb49-96743a01f569 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:42.731035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46047732-4083-465f-bf2a-796881f18cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:42.732409 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46047732-4083-465f-bf2a-796881f18cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:42.733170 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46047732-4083-465f-bf2a-796881f18cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:42.733455 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46047732-4083-465f-bf2a-796881f18cc5 None admin] PATH_INFO: `/v3/users/a59960e5047d40518f4471b4de5aae65` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:42.734728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46047732-4083-465f-bf2a-796881f18cc5 None admin] RBAC: Authorizing `identity:get_user(user_id=a59960e5047d40518f4471b4de5aae65)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:42.736730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46047732-4083-465f-bf2a-796881f18cc5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:42.738451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 79/155] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:42 2022] GET /identity/v3/users/a59960e5047d40518f4471b4de5aae65 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:42.744298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46047732-4083-465f-bf2a-796881f18cc5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:42.816471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 77/156] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:42.868287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46047732-4083-465f-bf2a-796881f18cc5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:42.913511 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4b82e-e6c6-4f82-a4c6-77440251911f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:42.914176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4b82e-e6c6-4f82-a4c6-77440251911f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:42.914469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4b82e-e6c6-4f82-a4c6-77440251911f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:42.914755 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4b82e-e6c6-4f82-a4c6-77440251911f None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:42.917510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4b82e-e6c6-4f82-a4c6-77440251911f None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:42.919099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4b82e-e6c6-4f82-a4c6-77440251911f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:42.921187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 80/157] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:42 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:42.927102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a33408-768a-405e-b3a0-31553d3ccfe3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:42.931492 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4b82e-e6c6-4f82-a4c6-77440251911f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:42.952901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:43.071053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4b82e-e6c6-4f82-a4c6-77440251911f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:43.102804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a33408-768a-405e-b3a0-31553d3ccfe3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:43.114450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d6772-77c8-4838-80ee-4fe110f93c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:43.115122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d6772-77c8-4838-80ee-4fe110f93c50 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:43.115460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d6772-77c8-4838-80ee-4fe110f93c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:43.115744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d6772-77c8-4838-80ee-4fe110f93c50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:43.126900 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d6772-77c8-4838-80ee-4fe110f93c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:43.139558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-c32d6772-77c8-4838-80ee-4fe110f93c50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:43.141239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:43.142194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:43.143390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 81/158] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:43.148720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d6772-77c8-4838-80ee-4fe110f93c50 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:43.172049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74e5dd-6d77-4f85-8182-8d9749b57d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:43.172763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74e5dd-6d77-4f85-8182-8d9749b57d3a None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:43.173051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74e5dd-6d77-4f85-8182-8d9749b57d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:43.173336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74e5dd-6d77-4f85-8182-8d9749b57d3a None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2/users/a59960e5047d40518f4471b4de5aae65/roles/84c485f3441a4653b579e9ab39d98a95` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:43.187998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74e5dd-6d77-4f85-8182-8d9749b57d3a None admin] RBAC: Authorizing `identity:create_grant(project_id=232b566033974d52b87f5e44eb52f5a2, user_id=a59960e5047d40518f4471b4de5aae65, role_id=84c485f3441a4653b579e9ab39d98a95)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:43.189495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74e5dd-6d77-4f85-8182-8d9749b57d3a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:43.219020 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 78/159] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:23:42 2022] PUT /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2/users/a59960e5047d40518f4471b4de5aae65/roles/84c485f3441a4653b579e9ab39d98a95 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:23:43.268422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:43.286626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d6772-77c8-4838-80ee-4fe110f93c50 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:43.336488 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a25dc-324c-4cb8-a546-2d40346b83eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:43.337559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a25dc-324c-4cb8-a546-2d40346b83eb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:43.338044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a25dc-324c-4cb8-a546-2d40346b83eb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:43.338563 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a25dc-324c-4cb8-a546-2d40346b83eb None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:43.339545 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a25dc-324c-4cb8-a546-2d40346b83eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:43.340056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a25dc-324c-4cb8-a546-2d40346b83eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:43.341775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a25dc-324c-4cb8-a546-2d40346b83eb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:43.352680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 82/160] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:43.358687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74e5dd-6d77-4f85-8182-8d9749b57d3a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:43.500952 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74e5dd-6d77-4f85-8182-8d9749b57d3a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:43.545782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fc1c5-d423-4320-b42f-35d88ef16e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:43.546513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fc1c5-d423-4320-b42f-35d88ef16e9d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:43.546768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fc1c5-d423-4320-b42f-35d88ef16e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:43.546984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fc1c5-d423-4320-b42f-35d88ef16e9d None admin] PATH_INFO: `/v3/users/5b4524f7f2f6410e983dadf2d798c568` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:43.548253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fc1c5-d423-4320-b42f-35d88ef16e9d None admin] RBAC: Authorizing `identity:get_user(user_id=5b4524f7f2f6410e983dadf2d798c568)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:43.549715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fc1c5-d423-4320-b42f-35d88ef16e9d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:43.551375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 79/161] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:43 2022] GET /identity/v3/users/5b4524f7f2f6410e983dadf2d798c568 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:43.557060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a25dc-324c-4cb8-a546-2d40346b83eb None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:43.649984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308f633-4dc6-412c-8014-128881feb154 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:43.650269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308f633-4dc6-412c-8014-128881feb154 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:43.650469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308f633-4dc6-412c-8014-128881feb154 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:43.651865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 80/162] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:43.664791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc0a19-b946-4c8a-a590-d698a0347f53 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:43.665109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc0a19-b946-4c8a-a590-d698a0347f53 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:43.665355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc0a19-b946-4c8a-a590-d698a0347f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:43.698639 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a9bc0a19-b946-4c8a-a590-d698a0347f53 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:43.699898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a9bc0a19-b946-4c8a-a590-d698a0347f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:43.707386 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a25dc-324c-4cb8-a546-2d40346b83eb None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:43.756529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab60ea2-d529-472f-bf1d-f2f94aaf78b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:43.757519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab60ea2-d529-472f-bf1d-f2f94aaf78b4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:43.758013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab60ea2-d529-472f-bf1d-f2f94aaf78b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:43.758488 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab60ea2-d529-472f-bf1d-f2f94aaf78b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:43.762884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:43.778537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:43.807510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 81/163] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:43.811013 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a84404-66ca-4f93-b981-f4fdefc7f2fa None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:43.811332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a84404-66ca-4f93-b981-f4fdefc7f2fa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:43.811617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a84404-66ca-4f93-b981-f4fdefc7f2fa None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:43.812919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 82/164] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:43.823051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449e4d1-7c88-4296-b734-df838fbd8090 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:43.823364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449e4d1-7c88-4296-b734-df838fbd8090 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:43.823647 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449e4d1-7c88-4296-b734-df838fbd8090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:43.864253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5449e4d1-7c88-4296-b734-df838fbd8090 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:43.865468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5449e4d1-7c88-4296-b734-df838fbd8090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:43.896765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab60ea2-d529-472f-bf1d-f2f94aaf78b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:43.898362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab60ea2-d529-472f-bf1d-f2f94aaf78b4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:43.975861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 83/165] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:43.976907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449e4d1-7c88-4296-b734-df838fbd8090 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:44.072835 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 83/166] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:44.073994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab60ea2-d529-472f-bf1d-f2f94aaf78b4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:44.082996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449e4d1-7c88-4296-b734-df838fbd8090 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:44.096845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:44.120303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7ac80-3510-4b3a-9a2a-abd679c338ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:44.121072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7ac80-3510-4b3a-9a2a-abd679c338ec None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:44.121422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7ac80-3510-4b3a-9a2a-abd679c338ec None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:44.121780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7ac80-3510-4b3a-9a2a-abd679c338ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:44.136829 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:44.137748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7ac80-3510-4b3a-9a2a-abd679c338ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-41b7ac80-3510-4b3a-9a2a-abd679c338ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:44.147510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:44.148420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:44.149430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 84/167] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:23:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:44.150646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7ac80-3510-4b3a-9a2a-abd679c338ec None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:44.249700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab60ea2-d529-472f-bf1d-f2f94aaf78b4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:44.253239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7ac80-3510-4b3a-9a2a-abd679c338ec None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:44.292279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f017a2-d5a0-4a48-a7ce-eaca9049e82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:44.292932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f017a2-d5a0-4a48-a7ce-eaca9049e82c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:44.293218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f017a2-d5a0-4a48-a7ce-eaca9049e82c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:44.293498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f017a2-d5a0-4a48-a7ce-eaca9049e82c None admin] PATH_INFO: `/v3/projects/ad0169ffda62481b93f959e15f033d13` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:44.294966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f017a2-d5a0-4a48-a7ce-eaca9049e82c None admin] RBAC: Authorizing `identity:get_project(project_id=ad0169ffda62481b93f959e15f033d13)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:44.297155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f017a2-d5a0-4a48-a7ce-eaca9049e82c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:44.299055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 85/168] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:44 2022] GET /identity/v3/projects/ad0169ffda62481b93f959e15f033d13 => generated 312 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:44.299632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f017a2-d5a0-4a48-a7ce-eaca9049e82c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:44.309335 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f87d3d-3f38-4fdc-b1f2-3a6e0c83b49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:44.310596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f87d3d-3f38-4fdc-b1f2-3a6e0c83b49c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:44.310881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f87d3d-3f38-4fdc-b1f2-3a6e0c83b49c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:44.311188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f87d3d-3f38-4fdc-b1f2-3a6e0c83b49c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:44.335178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f87d3d-3f38-4fdc-b1f2-3a6e0c83b49c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-14f87d3d-3f38-4fdc-b1f2-3a6e0c83b49c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:44.340151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:44.341532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:44.342379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:44.342379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:44.342379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:44.342379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:44.342379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:44.342379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:44.342379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:44.342379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:44.342379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:44.342379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:44.342379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:23:44.342379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:44.343676 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 84/169] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:23:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:44.345041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f87d3d-3f38-4fdc-b1f2-3a6e0c83b49c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:44.369001 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:44.431992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f017a2-d5a0-4a48-a7ce-eaca9049e82c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:44.472424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09172441-30ce-49d9-b116-de1df12e8b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:44.473424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09172441-30ce-49d9-b116-de1df12e8b88 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:44.473902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09172441-30ce-49d9-b116-de1df12e8b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:44.474471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09172441-30ce-49d9-b116-de1df12e8b88 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:44.475292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09172441-30ce-49d9-b116-de1df12e8b88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:44.475666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09172441-30ce-49d9-b116-de1df12e8b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:44.477271 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09172441-30ce-49d9-b116-de1df12e8b88 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:44.496463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 86/170] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:44.497746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09172441-30ce-49d9-b116-de1df12e8b88 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:44.501126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:44.506548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f87d3d-3f38-4fdc-b1f2-3a6e0c83b49c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:44.564717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f75ad2-17ee-4d80-8190-26b06d7a70f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:44.565350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f75ad2-17ee-4d80-8190-26b06d7a70f7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:44.565636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f75ad2-17ee-4d80-8190-26b06d7a70f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:44.565912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f75ad2-17ee-4d80-8190-26b06d7a70f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:44.566852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f75ad2-17ee-4d80-8190-26b06d7a70f7 None admin] RBAC: Adding query filter params (role.id=d8e3b1d89cfa4e0fb43b360a31b56605, scope.project.id=ad0169ffda62481b93f959e15f033d13, user.id=5b4524f7f2f6410e983dadf2d798c568) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:44.567011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f75ad2-17ee-4d80-8190-26b06d7a70f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:44.568505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f75ad2-17ee-4d80-8190-26b06d7a70f7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:44.583513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 85/171] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:44 2022] GET /identity/v3/role_assignments?user.id=5b4524f7f2f6410e983dadf2d798c568&scope.project.id=ad0169ffda62481b93f959e15f033d13&role.id=d8e3b1d89cfa4e0fb43b360a31b56605 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:44.584376 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f75ad2-17ee-4d80-8190-26b06d7a70f7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:44.627379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09172441-30ce-49d9-b116-de1df12e8b88 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:44.629527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:44.674935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff68a3-6271-49da-b33f-d7399c2183e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:44.675976 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff68a3-6271-49da-b33f-d7399c2183e1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:44.676543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff68a3-6271-49da-b33f-d7399c2183e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:44.677000 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff68a3-6271-49da-b33f-d7399c2183e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:44.678116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff68a3-6271-49da-b33f-d7399c2183e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:44.678624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff68a3-6271-49da-b33f-d7399c2183e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:44.680281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff68a3-6271-49da-b33f-d7399c2183e1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:44.690051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 87/172] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:23:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:44.696370 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff68a3-6271-49da-b33f-d7399c2183e1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:44.841777 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f75ad2-17ee-4d80-8190-26b06d7a70f7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:44.874771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff68a3-6271-49da-b33f-d7399c2183e1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:44.909405 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ec9dd-ed16-4a2a-b61f-097a4dc54227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:44.910062 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ec9dd-ed16-4a2a-b61f-097a4dc54227 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:44.910341 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ec9dd-ed16-4a2a-b61f-097a4dc54227 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:44.910614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ec9dd-ed16-4a2a-b61f-097a4dc54227 None admin] PATH_INFO: `/v3/users/aae3d76a87c24d97b28248f570423938` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:44.912789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ec9dd-ed16-4a2a-b61f-097a4dc54227 None admin] RBAC: Authorizing `identity:get_user(user_id=aae3d76a87c24d97b28248f570423938)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:44.914252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ec9dd-ed16-4a2a-b61f-097a4dc54227 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:44.915977 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 86/173] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:44 2022] GET /identity/v3/users/aae3d76a87c24d97b28248f570423938 => generated 247 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:44.922274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ec9dd-ed16-4a2a-b61f-097a4dc54227 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:44.944718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f40b66-6083-47d5-b507-45f25f900bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:44.945377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40b66-6083-47d5-b507-45f25f900bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:44.945661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40b66-6083-47d5-b507-45f25f900bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:44.945938 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40b66-6083-47d5-b507-45f25f900bb9 None admin] PATH_INFO: `/v3/users/5b4524f7f2f6410e983dadf2d798c568` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:44.947280 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f40b66-6083-47d5-b507-45f25f900bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=5b4524f7f2f6410e983dadf2d798c568)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:44.948765 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f40b66-6083-47d5-b507-45f25f900bb9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:44.953697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 88/174] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:44 2022] GET /identity/v3/users/5b4524f7f2f6410e983dadf2d798c568 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:44.960963 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f40b66-6083-47d5-b507-45f25f900bb9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:44.990318 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:45.030583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:45.105233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ec9dd-ed16-4a2a-b61f-097a4dc54227 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:45.168439 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92d2ee-8453-45f3-bc74-2576423dae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:45.169683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92d2ee-8453-45f3-bc74-2576423dae4e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:45.170022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92d2ee-8453-45f3-bc74-2576423dae4e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:45.170305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92d2ee-8453-45f3-bc74-2576423dae4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:45.197163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f40b66-6083-47d5-b507-45f25f900bb9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:45.296607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92d2ee-8453-45f3-bc74-2576423dae4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:45.297613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46465916-12dc-40d2-9c58-712c3f5aa4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:45.298222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92d2ee-8453-45f3-bc74-2576423dae4e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:45.298711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46465916-12dc-40d2-9c58-712c3f5aa4ac None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:45.299227 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46465916-12dc-40d2-9c58-712c3f5aa4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:45.299716 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46465916-12dc-40d2-9c58-712c3f5aa4ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:45.344377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:45.404180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 87/175] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:45.412178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92d2ee-8453-45f3-bc74-2576423dae4e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:45.507681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46465916-12dc-40d2-9c58-712c3f5aa4ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:45.509398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46465916-12dc-40d2-9c58-712c3f5aa4ac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:45.567594 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:45.581242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92d2ee-8453-45f3-bc74-2576423dae4e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:45.627600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:45.646660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01497f5e-f609-41db-8844-6304c54fcfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:45.647590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01497f5e-f609-41db-8844-6304c54fcfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:45.648019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01497f5e-f609-41db-8844-6304c54fcfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:45.648441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01497f5e-f609-41db-8844-6304c54fcfa3 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:45.651414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01497f5e-f609-41db-8844-6304c54fcfa3 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:45.653036 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01497f5e-f609-41db-8844-6304c54fcfa3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:45.655365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 88/176] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:45 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:45.657871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6ff0f-3c5a-428c-98f0-99bdf8ee144c None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:45.658967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6ff0f-3c5a-428c-98f0-99bdf8ee144c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:45.659574 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6ff0f-3c5a-428c-98f0-99bdf8ee144c None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:45.662337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 89/177] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:45.667931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6ff0f-3c5a-428c-98f0-99bdf8ee144c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:45.698884 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 89/178] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 740 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:45.701052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f25a8-c8be-46e2-bdc9-e0d1086f525c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:45.702409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f25a8-c8be-46e2-bdc9-e0d1086f525c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:45.703059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f25a8-c8be-46e2-bdc9-e0d1086f525c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:45.733381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a38f25a8-c8be-46e2-bdc9-e0d1086f525c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:45.736140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a38f25a8-c8be-46e2-bdc9-e0d1086f525c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:45.857642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6ff0f-3c5a-428c-98f0-99bdf8ee144c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:45.884668 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 90/179] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:45.885422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f25a8-c8be-46e2-bdc9-e0d1086f525c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:45.910730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aff615-0ec5-4a30-81b2-78f6ad9a72ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:45.911680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aff615-0ec5-4a30-81b2-78f6ad9a72ae None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:45.912162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aff615-0ec5-4a30-81b2-78f6ad9a72ae None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:45.912638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aff615-0ec5-4a30-81b2-78f6ad9a72ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:45.913644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aff615-0ec5-4a30-81b2-78f6ad9a72ae None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=232b566033974d52b87f5e44eb52f5a2, user.id=aae3d76a87c24d97b28248f570423938) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:45.914146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aff615-0ec5-4a30-81b2-78f6ad9a72ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:45.915993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aff615-0ec5-4a30-81b2-78f6ad9a72ae None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:45.927669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 90/180] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:45 2022] GET /identity/v3/role_assignments?user.id=aae3d76a87c24d97b28248f570423938&scope.project.id=232b566033974d52b87f5e44eb52f5a2&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:45.937014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aff615-0ec5-4a30-81b2-78f6ad9a72ae None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:46.040925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:46.052931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:46.093822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aff615-0ec5-4a30-81b2-78f6ad9a72ae None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:46.135905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b435a8-df2a-474c-95d3-5d6fdc45d2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:46.138834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b435a8-df2a-474c-95d3-5d6fdc45d2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.139658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b435a8-df2a-474c-95d3-5d6fdc45d2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.140140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b435a8-df2a-474c-95d3-5d6fdc45d2c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.153723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f25a8-c8be-46e2-bdc9-e0d1086f525c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:46.154889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b435a8-df2a-474c-95d3-5d6fdc45d2c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-80b435a8-df2a-474c-95d3-5d6fdc45d2c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:46.162144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:46.163016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:46.163842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 91/181] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:23:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:46.164982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2fa06-8527-407f-a315-1f3065fde28d None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.165152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2fa06-8527-407f-a315-1f3065fde28d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.170811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2fa06-8527-407f-a315-1f3065fde28d None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.170811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 92/182] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:46.171944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2fa06-8527-407f-a315-1f3065fde28d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:46.207369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713edaf8-3d85-4f0d-b884-6c41bdc91421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:46.208366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713edaf8-3d85-4f0d-b884-6c41bdc91421 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.208731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713edaf8-3d85-4f0d-b884-6c41bdc91421 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.208924 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713edaf8-3d85-4f0d-b884-6c41bdc91421 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.210816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713edaf8-3d85-4f0d-b884-6c41bdc91421 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:46.212315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713edaf8-3d85-4f0d-b884-6c41bdc91421 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:46.214136 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 91/183] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:45 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:46.215127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d063a82-f1ab-4e98-96d3-17407e1e468b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.215498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d063a82-f1ab-4e98-96d3-17407e1e468b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.216550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d063a82-f1ab-4e98-96d3-17407e1e468b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.235873 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1d063a82-f1ab-4e98-96d3-17407e1e468b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:46.237071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1d063a82-f1ab-4e98-96d3-17407e1e468b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:46.285100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2fa06-8527-407f-a315-1f3065fde28d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:46.307471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:46.316968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 92/184] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:46.318320 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e4440-d93d-4214-aa8d-1539a0791bc8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.318616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e4440-d93d-4214-aa8d-1539a0791bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.318893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e4440-d93d-4214-aa8d-1539a0791bc8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.320242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 93/185] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:46.324292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fa9a1-d54c-4f11-99e1-151439fafa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:46.324893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e4440-d93d-4214-aa8d-1539a0791bc8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:46.325750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fa9a1-d54c-4f11-99e1-151439fafa53 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.326055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fa9a1-d54c-4f11-99e1-151439fafa53 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.326329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fa9a1-d54c-4f11-99e1-151439fafa53 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.327047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fa9a1-d54c-4f11-99e1-151439fafa53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:46.327380 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fa9a1-d54c-4f11-99e1-151439fafa53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:46.328775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fa9a1-d54c-4f11-99e1-151439fafa53 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:46.338171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 93/186] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:46.339526 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f94e1-d174-4aca-9862-8098f31aede8 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.339809 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f94e1-d174-4aca-9862-8098f31aede8 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.340085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f94e1-d174-4aca-9862-8098f31aede8 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.342073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 94/187] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:46.343248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09797b3a-333e-4517-8009-35d7cf0ec28a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.343632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09797b3a-333e-4517-8009-35d7cf0ec28a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.343916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09797b3a-333e-4517-8009-35d7cf0ec28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.361785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-09797b3a-333e-4517-8009-35d7cf0ec28a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:46.362987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-09797b3a-333e-4517-8009-35d7cf0ec28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:46.392276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:46.431137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 95/188] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:46.431980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09797b3a-333e-4517-8009-35d7cf0ec28a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:46.441244 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e4440-d93d-4214-aa8d-1539a0791bc8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:46.476939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4337e5f-024e-4f19-a3aa-41cdab89a8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:46.477557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4337e5f-024e-4f19-a3aa-41cdab89a8ff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.477838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4337e5f-024e-4f19-a3aa-41cdab89a8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.478114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4337e5f-024e-4f19-a3aa-41cdab89a8ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.478893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4337e5f-024e-4f19-a3aa-41cdab89a8ff None admin] RBAC: Adding query filter params (role.id=0a56be0718ea495f8a241ae2447069cf, scope.project.id=232b566033974d52b87f5e44eb52f5a2, user.id=5b4524f7f2f6410e983dadf2d798c568) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:46.479242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4337e5f-024e-4f19-a3aa-41cdab89a8ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:46.480705 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4337e5f-024e-4f19-a3aa-41cdab89a8ff None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:46.487146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 94/189] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:46 2022] GET /identity/v3/role_assignments?user.id=5b4524f7f2f6410e983dadf2d798c568&scope.project.id=232b566033974d52b87f5e44eb52f5a2&role.id=0a56be0718ea495f8a241ae2447069cf => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:46.487919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4337e5f-024e-4f19-a3aa-41cdab89a8ff None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:46.537441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09797b3a-333e-4517-8009-35d7cf0ec28a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:46.553171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:46.579037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87982060-f30b-4823-8c98-d993a855767d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:46.579740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87982060-f30b-4823-8c98-d993a855767d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.580026 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87982060-f30b-4823-8c98-d993a855767d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.580306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87982060-f30b-4823-8c98-d993a855767d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.592776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87982060-f30b-4823-8c98-d993a855767d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-87982060-f30b-4823-8c98-d993a855767d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:46.597831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:46.598730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:46.599908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 96/190] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:23:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:46.600559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb4b86-9eb3-46b7-9870-41152529fb0b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.600559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb4b86-9eb3-46b7-9870-41152529fb0b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.600949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb4b86-9eb3-46b7-9870-41152529fb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.618107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4337e5f-024e-4f19-a3aa-41cdab89a8ff None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:46.620941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-19bb4b86-9eb3-46b7-9870-41152529fb0b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:46.622699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-19bb4b86-9eb3-46b7-9870-41152529fb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:46.644496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:46.658044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246db2d2-e14e-479d-a69c-773ad9301ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:46.661049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246db2d2-e14e-479d-a69c-773ad9301ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.661049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246db2d2-e14e-479d-a69c-773ad9301ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.661049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246db2d2-e14e-479d-a69c-773ad9301ce4 None admin] PATH_INFO: `/v3/users/629e80c75d8c4c97b4d6e790d84b7b90` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.678688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246db2d2-e14e-479d-a69c-773ad9301ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=629e80c75d8c4c97b4d6e790d84b7b90)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:46.681260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246db2d2-e14e-479d-a69c-773ad9301ce4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:46.682082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 95/191] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:46 2022] GET /identity/v3/users/629e80c75d8c4c97b4d6e790d84b7b90 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:46.682672 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246db2d2-e14e-479d-a69c-773ad9301ce4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:46.705937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 97/192] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:46.706788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb4b86-9eb3-46b7-9870-41152529fb0b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:46.796233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246db2d2-e14e-479d-a69c-773ad9301ce4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:46.815617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb4b86-9eb3-46b7-9870-41152529fb0b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:46.817117 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:46.829653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d361797-95e4-4a56-91bf-8a06d95df3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:46.830293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d361797-95e4-4a56-91bf-8a06d95df3db None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.830577 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d361797-95e4-4a56-91bf-8a06d95df3db None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.830869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d361797-95e4-4a56-91bf-8a06d95df3db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.832039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d361797-95e4-4a56-91bf-8a06d95df3db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:46.833482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d361797-95e4-4a56-91bf-8a06d95df3db None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:46.835262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 96/193] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:46.835861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d361797-95e4-4a56-91bf-8a06d95df3db None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:46.854143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e4880-2030-4465-8eea-8ddecbde89aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:46.854772 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e4880-2030-4465-8eea-8ddecbde89aa None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.855060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e4880-2030-4465-8eea-8ddecbde89aa None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.855371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e4880-2030-4465-8eea-8ddecbde89aa None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:46.856092 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e4880-2030-4465-8eea-8ddecbde89aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:46.856471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e4880-2030-4465-8eea-8ddecbde89aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:46.858661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e4880-2030-4465-8eea-8ddecbde89aa None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:46.866931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 98/194] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:23:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:46.867894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e4880-2030-4465-8eea-8ddecbde89aa None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:46.886974 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:46.936616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d361797-95e4-4a56-91bf-8a06d95df3db None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:46.966615 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e4880-2030-4465-8eea-8ddecbde89aa None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:46.973923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0d7a6-a14f-4671-9e88-7fcdbcc7c396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:46.974547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0d7a6-a14f-4671-9e88-7fcdbcc7c396 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:46.974831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0d7a6-a14f-4671-9e88-7fcdbcc7c396 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:46.975130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0d7a6-a14f-4671-9e88-7fcdbcc7c396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:47.000625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261cc30-0ce2-4798-b5a5-9a8603f64c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:47.001277 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261cc30-0ce2-4798-b5a5-9a8603f64c7e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:47.001564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261cc30-0ce2-4798-b5a5-9a8603f64c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:47.001855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261cc30-0ce2-4798-b5a5-9a8603f64c7e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:47.013558 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261cc30-0ce2-4798-b5a5-9a8603f64c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-d261cc30-0ce2-4798-b5a5-9a8603f64c7e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:47.019955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:47.020815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:47.021793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 99/195] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:47.023382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261cc30-0ce2-4798-b5a5-9a8603f64c7e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:47.044394 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0d7a6-a14f-4671-9e88-7fcdbcc7c396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:47.045861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0d7a6-a14f-4671-9e88-7fcdbcc7c396 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:47.047019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:47.093511 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:47.122890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261cc30-0ce2-4798-b5a5-9a8603f64c7e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:47.143899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 97/196] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:47.144728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0d7a6-a14f-4671-9e88-7fcdbcc7c396 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:47.158541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb9607-126c-49d2-9c42-036f72560902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:47.159187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb9607-126c-49d2-9c42-036f72560902 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:47.159466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb9607-126c-49d2-9c42-036f72560902 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:47.159744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb9607-126c-49d2-9c42-036f72560902 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:47.160418 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb9607-126c-49d2-9c42-036f72560902 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:47.161892 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb9607-126c-49d2-9c42-036f72560902 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:47.182015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 100/197] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:23:47 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:23:47.182831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb9607-126c-49d2-9c42-036f72560902 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:47.259677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0d7a6-a14f-4671-9e88-7fcdbcc7c396 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:47.289498 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:47.301647 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7552a3-9a2a-452f-99e0-6dc3b6115285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:47.302407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7552a3-9a2a-452f-99e0-6dc3b6115285 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:47.302760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7552a3-9a2a-452f-99e0-6dc3b6115285 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:47.303142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7552a3-9a2a-452f-99e0-6dc3b6115285 None admin] PATH_INFO: `/v3/users/629e80c75d8c4c97b4d6e790d84b7b90` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:47.304405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7552a3-9a2a-452f-99e0-6dc3b6115285 None admin] RBAC: Authorizing `identity:get_user(user_id=629e80c75d8c4c97b4d6e790d84b7b90)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:47.305897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7552a3-9a2a-452f-99e0-6dc3b6115285 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:47.307843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 98/198] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:47 2022] GET /identity/v3/users/629e80c75d8c4c97b4d6e790d84b7b90 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:47.308486 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7552a3-9a2a-452f-99e0-6dc3b6115285 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:47.322247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb9607-126c-49d2-9c42-036f72560902 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:47.335040 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:47.362341 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63d7d8-0850-4099-81ed-9f8169cc1f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:47.362969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63d7d8-0850-4099-81ed-9f8169cc1f1e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:47.363351 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63d7d8-0850-4099-81ed-9f8169cc1f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:47.363635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63d7d8-0850-4099-81ed-9f8169cc1f1e None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:47.364361 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63d7d8-0850-4099-81ed-9f8169cc1f1e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:47.364673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63d7d8-0850-4099-81ed-9f8169cc1f1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:47.366060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63d7d8-0850-4099-81ed-9f8169cc1f1e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:47.374961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 101/199] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:47.375814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63d7d8-0850-4099-81ed-9f8169cc1f1e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:47.438361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7552a3-9a2a-452f-99e0-6dc3b6115285 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:47.495019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c24d0-9e1b-41b6-9fdd-088ea520eca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:47.495718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c24d0-9e1b-41b6-9fdd-088ea520eca8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:47.496010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c24d0-9e1b-41b6-9fdd-088ea520eca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:47.496286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c24d0-9e1b-41b6-9fdd-088ea520eca8 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:47.497908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c24d0-9e1b-41b6-9fdd-088ea520eca8 None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:47.499607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c24d0-9e1b-41b6-9fdd-088ea520eca8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:47.501413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 99/200] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:47 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:47.501999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c24d0-9e1b-41b6-9fdd-088ea520eca8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:47.503955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63d7d8-0850-4099-81ed-9f8169cc1f1e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:47.541919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e1e42-926e-407b-8340-2527196313c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:47.542474 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e1e42-926e-407b-8340-2527196313c0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:47.542758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e1e42-926e-407b-8340-2527196313c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:47.543039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e1e42-926e-407b-8340-2527196313c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:47.556181 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:47.615955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:47.631596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e1e42-926e-407b-8340-2527196313c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:47.633367 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e1e42-926e-407b-8340-2527196313c0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:47.637682 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c24d0-9e1b-41b6-9fdd-088ea520eca8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:47.685258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00f559-a8c5-4770-9533-83220ec69621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:47.686192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00f559-a8c5-4770-9533-83220ec69621 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:47.686541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00f559-a8c5-4770-9533-83220ec69621 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:47.687041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00f559-a8c5-4770-9533-83220ec69621 None admin] PATH_INFO: `/v3/users/a59960e5047d40518f4471b4de5aae65` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:47.688650 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00f559-a8c5-4770-9533-83220ec69621 None admin] RBAC: Authorizing `identity:get_user(user_id=a59960e5047d40518f4471b4de5aae65)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:47.690179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00f559-a8c5-4770-9533-83220ec69621 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:47.692905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 100/201] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:47 2022] GET /identity/v3/users/a59960e5047d40518f4471b4de5aae65 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:47.693971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00f559-a8c5-4770-9533-83220ec69621 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:47.756783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 102/202] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:47.759783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e1e42-926e-407b-8340-2527196313c0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:47.810932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00f559-a8c5-4770-9533-83220ec69621 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:47.821042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:47.853316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857f58d-1ce8-47f3-81e0-6772c384d6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:47.854351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f58d-1ce8-47f3-81e0-6772c384d6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:47.854638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f58d-1ce8-47f3-81e0-6772c384d6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:47.854914 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f58d-1ce8-47f3-81e0-6772c384d6e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:47.856103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857f58d-1ce8-47f3-81e0-6772c384d6e4 None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=731d452341b64364b192b87761b43850, user.id=629e80c75d8c4c97b4d6e790d84b7b90) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:47.856958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857f58d-1ce8-47f3-81e0-6772c384d6e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:47.858781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857f58d-1ce8-47f3-81e0-6772c384d6e4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:47.867593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 101/203] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:47 2022] GET /identity/v3/role_assignments?user.id=629e80c75d8c4c97b4d6e790d84b7b90&scope.project.id=731d452341b64364b192b87761b43850&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:47.868908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857f58d-1ce8-47f3-81e0-6772c384d6e4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:47.908190 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e1e42-926e-407b-8340-2527196313c0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:47.923497 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:48.032301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857f58d-1ce8-47f3-81e0-6772c384d6e4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:48.033429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4682e-8d2e-42fc-b30a-6dc4293a95b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:48.034147 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4682e-8d2e-42fc-b30a-6dc4293a95b6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:48.034433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4682e-8d2e-42fc-b30a-6dc4293a95b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:48.034711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4682e-8d2e-42fc-b30a-6dc4293a95b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:48.101843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eab051-b047-4033-9c0e-c23ea8a67a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:48.102407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eab051-b047-4033-9c0e-c23ea8a67a9f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:48.102691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eab051-b047-4033-9c0e-c23ea8a67a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:48.102969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eab051-b047-4033-9c0e-c23ea8a67a9f None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:48.105165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eab051-b047-4033-9c0e-c23ea8a67a9f None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:48.106778 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eab051-b047-4033-9c0e-c23ea8a67a9f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:48.113059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 102/204] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:47 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:48.118817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eab051-b047-4033-9c0e-c23ea8a67a9f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:48.182091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4682e-8d2e-42fc-b30a-6dc4293a95b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:48.183624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4682e-8d2e-42fc-b30a-6dc4293a95b6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:48.266741 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:48.283543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:48.308452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eab051-b047-4033-9c0e-c23ea8a67a9f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:48.368554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 103/205] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:48.371896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d85e7-501f-4179-946e-f0745a5d96b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:48.372675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d85e7-501f-4179-946e-f0745a5d96b7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:48.373032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d85e7-501f-4179-946e-f0745a5d96b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:48.373383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d85e7-501f-4179-946e-f0745a5d96b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:48.374248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d85e7-501f-4179-946e-f0745a5d96b7 None admin] RBAC: Adding query filter params (role.id=0a56be0718ea495f8a241ae2447069cf, scope.project.id=731d452341b64364b192b87761b43850, user.id=629e80c75d8c4c97b4d6e790d84b7b90) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:48.374648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d85e7-501f-4179-946e-f0745a5d96b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:48.376006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4682e-8d2e-42fc-b30a-6dc4293a95b6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:48.378999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d85e7-501f-4179-946e-f0745a5d96b7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:48.393469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 103/206] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:48 2022] GET /identity/v3/role_assignments?user.id=629e80c75d8c4c97b4d6e790d84b7b90&scope.project.id=731d452341b64364b192b87761b43850&role.id=0a56be0718ea495f8a241ae2447069cf => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:48.530508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4682e-8d2e-42fc-b30a-6dc4293a95b6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:48.572309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ffa088-c146-46f2-8936-02385ce09240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:48.572967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffa088-c146-46f2-8936-02385ce09240 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:48.573255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffa088-c146-46f2-8936-02385ce09240 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:48.573535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffa088-c146-46f2-8936-02385ce09240 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:48.574723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ffa088-c146-46f2-8936-02385ce09240 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:48.576589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ffa088-c146-46f2-8936-02385ce09240 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:48.578630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 104/207] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:48 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:48.584633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d85e7-501f-4179-946e-f0745a5d96b7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:48.718849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d85e7-501f-4179-946e-f0745a5d96b7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:48.763635 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7da2c-e307-40ef-af1d-c62d8de9297f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:48.764254 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7da2c-e307-40ef-af1d-c62d8de9297f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:48.764539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7da2c-e307-40ef-af1d-c62d8de9297f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:48.764819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7da2c-e307-40ef-af1d-c62d8de9297f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:48.765612 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7da2c-e307-40ef-af1d-c62d8de9297f None admin] RBAC: Adding query filter params (role.id=84c485f3441a4653b579e9ab39d98a95, scope.project.id=232b566033974d52b87f5e44eb52f5a2, user.id=a59960e5047d40518f4471b4de5aae65) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:48.765928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7da2c-e307-40ef-af1d-c62d8de9297f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:48.767463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7da2c-e307-40ef-af1d-c62d8de9297f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:48.776850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 104/208] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:48 2022] GET /identity/v3/role_assignments?user.id=a59960e5047d40518f4471b4de5aae65&scope.project.id=232b566033974d52b87f5e44eb52f5a2&role.id=84c485f3441a4653b579e9ab39d98a95 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:49.459428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b38bc5-5176-4d35-948b-f4e6f7a3b277 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:49.460333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b38bc5-5176-4d35-948b-f4e6f7a3b277 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:49.460620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b38bc5-5176-4d35-948b-f4e6f7a3b277 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:49.462132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 105/209] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:49.467768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7772940-067c-4eb4-b892-18be465dd02c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:49.468192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7772940-067c-4eb4-b892-18be465dd02c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:49.468383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7772940-067c-4eb4-b892-18be465dd02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:49.488916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e7772940-067c-4eb4-b892-18be465dd02c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:49.490425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e7772940-067c-4eb4-b892-18be465dd02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:49.501263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:49.567192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 105/210] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:49.589540 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b38bc5-5176-4d35-948b-f4e6f7a3b277 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:49.603913 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:49.659011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e5816-208c-45d0-8c14-66a96a2951b9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:49.659344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e5816-208c-45d0-8c14-66a96a2951b9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:49.659608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e5816-208c-45d0-8c14-66a96a2951b9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:49.661048 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 106/211] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:23:49.666013 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4918a8-1208-4ad3-811e-ea989817c794 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:49.666304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4918a8-1208-4ad3-811e-ea989817c794 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:49.666587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4918a8-1208-4ad3-811e-ea989817c794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:49.685895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fe4918a8-1208-4ad3-811e-ea989817c794 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:49.687191 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fe4918a8-1208-4ad3-811e-ea989817c794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:49.706691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b38bc5-5176-4d35-948b-f4e6f7a3b277 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:49.746435 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5869ba-c491-4f66-96b0-348ab0c3fc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:49.747532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5869ba-c491-4f66-96b0-348ab0c3fc3e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:49.748015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5869ba-c491-4f66-96b0-348ab0c3fc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:49.748485 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5869ba-c491-4f66-96b0-348ab0c3fc3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:49.759620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5869ba-c491-4f66-96b0-348ab0c3fc3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:49.763393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 107/212] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-db5869ba-c491-4f66-96b0-348ab0c3fc3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:49.765923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:49.766765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:49.767591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:49.767591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:49.767591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:49.767591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:49.767591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:49.767591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:49.767591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:49.767591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:49.767591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:49.767591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:49.767591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:49.767591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:49.770102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 106/213] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:49.776378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4918a8-1208-4ad3-811e-ea989817c794 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:49.789602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5869ba-c491-4f66-96b0-348ab0c3fc3e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:49.857951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:49.864699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:49.887720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4918a8-1208-4ad3-811e-ea989817c794 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:49.899265 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5869ba-c491-4f66-96b0-348ab0c3fc3e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:49.925468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8644b09-28d4-4765-9c7b-627675b1d3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:49.926095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8644b09-28d4-4765-9c7b-627675b1d3ee None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:49.926380 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8644b09-28d4-4765-9c7b-627675b1d3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:49.926660 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8644b09-28d4-4765-9c7b-627675b1d3ee None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:49.927426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8644b09-28d4-4765-9c7b-627675b1d3ee None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:49.927938 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8644b09-28d4-4765-9c7b-627675b1d3ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:49.930433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8644b09-28d4-4765-9c7b-627675b1d3ee None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:49.937820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2ce79-d519-4142-98e1-5e4f9c1f7164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:49.938727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2ce79-d519-4142-98e1-5e4f9c1f7164 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:49.939236 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2ce79-d519-4142-98e1-5e4f9c1f7164 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:49.939704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2ce79-d519-4142-98e1-5e4f9c1f7164 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:49.941176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2ce79-d519-4142-98e1-5e4f9c1f7164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:49.943183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2ce79-d519-4142-98e1-5e4f9c1f7164 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:49.944298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 108/214] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:49.945461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8644b09-28d4-4765-9c7b-627675b1d3ee None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:49.951048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 107/215] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:49.952540 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2ce79-d519-4142-98e1-5e4f9c1f7164 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:50.059104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2ce79-d519-4142-98e1-5e4f9c1f7164 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:50.077089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8644b09-28d4-4765-9c7b-627675b1d3ee None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:50.095700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fdd82-7339-40ad-ae3a-af4fb0def204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:50.096628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fdd82-7339-40ad-ae3a-af4fb0def204 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:50.097136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fdd82-7339-40ad-ae3a-af4fb0def204 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:50.097605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fdd82-7339-40ad-ae3a-af4fb0def204 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:50.098490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fdd82-7339-40ad-ae3a-af4fb0def204 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:50.100167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fdd82-7339-40ad-ae3a-af4fb0def204 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:50.110818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:50.114557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1a766-9c7d-43a8-94f1-ff3764fc4acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:50.115404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1a766-9c7d-43a8-94f1-ff3764fc4acf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:50.115910 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1a766-9c7d-43a8-94f1-ff3764fc4acf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:50.116223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1a766-9c7d-43a8-94f1-ff3764fc4acf None admin] PATH_INFO: `/v3/users/5a44a48b9d034f9788579b849e36e758` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:50.121076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:50.126970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 108/216] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:23:49 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:23:50.136896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1a766-9c7d-43a8-94f1-ff3764fc4acf None admin] RBAC: Authorizing `identity:get_user(user_id=5a44a48b9d034f9788579b849e36e758)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:50.139676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1a766-9c7d-43a8-94f1-ff3764fc4acf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:50.141712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 109/217] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:49 2022] GET /identity/v3/users/5a44a48b9d034f9788579b849e36e758 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:50.147798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fdd82-7339-40ad-ae3a-af4fb0def204 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:50.217509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ebf5c-35ec-4e84-bded-9f0f7de2087e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:50.217807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ebf5c-35ec-4e84-bded-9f0f7de2087e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:50.218058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ebf5c-35ec-4e84-bded-9f0f7de2087e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:50.219712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 110/218] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:23:50.227593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e65d3-e153-4d3d-87ac-f84ea24e3013 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:50.227593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e65d3-e153-4d3d-87ac-f84ea24e3013 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:50.227972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e65d3-e153-4d3d-87ac-f84ea24e3013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:50.247273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ae8e65d3-e153-4d3d-87ac-f84ea24e3013 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:50.248810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ae8e65d3-e153-4d3d-87ac-f84ea24e3013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:50.279338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fdd82-7339-40ad-ae3a-af4fb0def204 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:50.317028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af93c5-7bcb-4286-99d5-2ca2dc95fcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:50.317596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af93c5-7bcb-4286-99d5-2ca2dc95fcfd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:50.317863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af93c5-7bcb-4286-99d5-2ca2dc95fcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:50.318137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af93c5-7bcb-4286-99d5-2ca2dc95fcfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:50.334961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 111/219] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:50.356688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e65d3-e153-4d3d-87ac-f84ea24e3013 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:50.390276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:50.397400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af93c5-7bcb-4286-99d5-2ca2dc95fcfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:50.399334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af93c5-7bcb-4286-99d5-2ca2dc95fcfd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:50.484768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e65d3-e153-4d3d-87ac-f84ea24e3013 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:50.518051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:50.526582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 109/220] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:50.532623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af93c5-7bcb-4286-99d5-2ca2dc95fcfd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:50.540361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d08aa3-d805-4aae-8455-bca46c92eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:50.541736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d08aa3-d805-4aae-8455-bca46c92eeea None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:50.542225 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d08aa3-d805-4aae-8455-bca46c92eeea None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:50.543112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d08aa3-d805-4aae-8455-bca46c92eeea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:50.558023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d08aa3-d805-4aae-8455-bca46c92eeea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-16d08aa3-d805-4aae-8455-bca46c92eeea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:50.565709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:50.566615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:50.567501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:50.567501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:50.567501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:50.567501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:50.567501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:50.567501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:50.567501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:50.567501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:50.567501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:50.567501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:50.567501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:50.567501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:50.568125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 112/221] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:23:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:50.573402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d08aa3-d805-4aae-8455-bca46c92eeea None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:50.643485 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:50.651787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af93c5-7bcb-4286-99d5-2ca2dc95fcfd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:50.688377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d08aa3-d805-4aae-8455-bca46c92eeea None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:50.724295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3a3b4-79f5-4c07-b0f7-a068e61e2197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:50.724920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3a3b4-79f5-4c07-b0f7-a068e61e2197 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:50.725204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3a3b4-79f5-4c07-b0f7-a068e61e2197 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:50.725482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3a3b4-79f5-4c07-b0f7-a068e61e2197 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:50.726204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3a3b4-79f5-4c07-b0f7-a068e61e2197 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:50.726574 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3a3b4-79f5-4c07-b0f7-a068e61e2197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:50.728057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3a3b4-79f5-4c07-b0f7-a068e61e2197 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:50.733834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a7f68-81d7-4477-8dd4-7c66d4d07eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:50.734631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a7f68-81d7-4477-8dd4-7c66d4d07eda None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:50.735021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a7f68-81d7-4477-8dd4-7c66d4d07eda None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:50.735435 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a7f68-81d7-4477-8dd4-7c66d4d07eda None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:50.736720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a7f68-81d7-4477-8dd4-7c66d4d07eda None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:50.737350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 113/222] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:50.738385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a7f68-81d7-4477-8dd4-7c66d4d07eda None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:50.738589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabed547-3d02-45f2-af5d-164c40c8dc9d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:50.738877 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabed547-3d02-45f2-af5d-164c40c8dc9d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:50.740709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 110/223] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:50 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:50.743490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabed547-3d02-45f2-af5d-164c40c8dc9d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:50.745045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 114/224] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:50.746021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a7f68-81d7-4477-8dd4-7c66d4d07eda None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:50.747480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabed547-3d02-45f2-af5d-164c40c8dc9d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:50.808295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:50.873381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabed547-3d02-45f2-af5d-164c40c8dc9d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:50.879457 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a7f68-81d7-4477-8dd4-7c66d4d07eda None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:50.916699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c8b67-cfe0-4407-bef4-c1708a919d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:50.917426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c8b67-cfe0-4407-bef4-c1708a919d2b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:50.917547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d45718-e73a-49f1-ad0c-58667b3d1b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:50.917754 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c8b67-cfe0-4407-bef4-c1708a919d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:50.917987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c8b67-cfe0-4407-bef4-c1708a919d2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:50.918145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d45718-e73a-49f1-ad0c-58667b3d1b69 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:50.918433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d45718-e73a-49f1-ad0c-58667b3d1b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:50.919343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c8b67-cfe0-4407-bef4-c1708a919d2b None admin] RBAC: Adding query filter params (role.id=d8e3b1d89cfa4e0fb43b360a31b56605, scope.project.id=731d452341b64364b192b87761b43850, user.id=5a44a48b9d034f9788579b849e36e758) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:50.919461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d45718-e73a-49f1-ad0c-58667b3d1b69 None admin] PATH_INFO: `/v3/users/629e80c75d8c4c97b4d6e790d84b7b90` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:50.919655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c8b67-cfe0-4407-bef4-c1708a919d2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:50.920608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d45718-e73a-49f1-ad0c-58667b3d1b69 None admin] RBAC: Authorizing `identity:get_user(user_id=629e80c75d8c4c97b4d6e790d84b7b90)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:50.921229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c8b67-cfe0-4407-bef4-c1708a919d2b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:50.922056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d45718-e73a-49f1-ad0c-58667b3d1b69 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:50.924030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 111/225] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:50 2022] GET /identity/v3/users/629e80c75d8c4c97b4d6e790d84b7b90 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:50.925177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99f96e-8bb3-4b52-ab5b-52db2bfe06e4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:50.925470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99f96e-8bb3-4b52-ab5b-52db2bfe06e4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:50.925751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99f96e-8bb3-4b52-ab5b-52db2bfe06e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:50.932171 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 115/226] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:50 2022] GET /identity/v3/role_assignments?user.id=5a44a48b9d034f9788579b849e36e758&scope.project.id=731d452341b64364b192b87761b43850&role.id=d8e3b1d89cfa4e0fb43b360a31b56605 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:50.932992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c8b67-cfe0-4407-bef4-c1708a919d2b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:50.963220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2a99f96e-8bb3-4b52-ab5b-52db2bfe06e4 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:50.964523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2a99f96e-8bb3-4b52-ab5b-52db2bfe06e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:51.009950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:51.064468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 112/227] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:51.087497 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99f96e-8bb3-4b52-ab5b-52db2bfe06e4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:51.159912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c8b67-cfe0-4407-bef4-c1708a919d2b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:51.183421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:51.201576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db27e7a-e4b1-4e53-b9d0-605e64b9ef98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:51.202226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db27e7a-e4b1-4e53-b9d0-605e64b9ef98 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:51.202508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db27e7a-e4b1-4e53-b9d0-605e64b9ef98 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:51.202786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db27e7a-e4b1-4e53-b9d0-605e64b9ef98 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:51.203981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db27e7a-e4b1-4e53-b9d0-605e64b9ef98 None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:51.205423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db27e7a-e4b1-4e53-b9d0-605e64b9ef98 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:51.207296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 116/228] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:50 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:51.212762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db27e7a-e4b1-4e53-b9d0-605e64b9ef98 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:51.270797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99f96e-8bb3-4b52-ab5b-52db2bfe06e4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:51.317662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db27e7a-e4b1-4e53-b9d0-605e64b9ef98 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:51.352471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74c2fa-0cf3-4a92-8a13-c83c50587749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:51.353195 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74c2fa-0cf3-4a92-8a13-c83c50587749 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:51.353484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74c2fa-0cf3-4a92-8a13-c83c50587749 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:51.353761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74c2fa-0cf3-4a92-8a13-c83c50587749 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850/users/629e80c75d8c4c97b4d6e790d84b7b90/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:51.355737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74c2fa-0cf3-4a92-8a13-c83c50587749 None admin] RBAC: Authorizing `identity:create_grant(project_id=731d452341b64364b192b87761b43850, user_id=629e80c75d8c4c97b4d6e790d84b7b90, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:51.357276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74c2fa-0cf3-4a92-8a13-c83c50587749 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:51.367561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4588c-7217-48ab-93b7-977a8cc15ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:51.368326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4588c-7217-48ab-93b7-977a8cc15ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:51.368610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4588c-7217-48ab-93b7-977a8cc15ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:51.368891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4588c-7217-48ab-93b7-977a8cc15ca1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:51.389121 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 117/229] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:23:51 2022] PUT /identity/v3/projects/731d452341b64364b192b87761b43850/users/629e80c75d8c4c97b4d6e790d84b7b90/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:23:51.404196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4588c-7217-48ab-93b7-977a8cc15ca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-51b4588c-7217-48ab-93b7-977a8cc15ca1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:51.409491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:51.410509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:51.411395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:51.411395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:51.411395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:51.411395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:51.411395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:51.411395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:51.411395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:51.411395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:51.411395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:51.411395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:51.411395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:23:51.411395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:51.411996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 113/230] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:23:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:23:51.416548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74c2fa-0cf3-4a92-8a13-c83c50587749 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:51.475479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:51.536165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74c2fa-0cf3-4a92-8a13-c83c50587749 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:51.576432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1faed-1e4b-4b7f-aa00-df6fedce60bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:51.577460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1faed-1e4b-4b7f-aa00-df6fedce60bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:51.577949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1faed-1e4b-4b7f-aa00-df6fedce60bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:51.578450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1faed-1e4b-4b7f-aa00-df6fedce60bb None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:51.579456 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1faed-1e4b-4b7f-aa00-df6fedce60bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:51.579969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1faed-1e4b-4b7f-aa00-df6fedce60bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:51.581631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1faed-1e4b-4b7f-aa00-df6fedce60bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:51.593463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 118/231] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:23:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:51.599282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4588c-7217-48ab-93b7-977a8cc15ca1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:51.633885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:51.727930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4588c-7217-48ab-93b7-977a8cc15ca1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:51.777623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151de64e-91da-434c-82aa-915f96dfdc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:51.779534 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151de64e-91da-434c-82aa-915f96dfdc79 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:51.780041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151de64e-91da-434c-82aa-915f96dfdc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:51.780515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151de64e-91da-434c-82aa-915f96dfdc79 None admin] PATH_INFO: `/v3/users/629e80c75d8c4c97b4d6e790d84b7b90` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:51.787828 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151de64e-91da-434c-82aa-915f96dfdc79 None admin] RBAC: Authorizing `identity:get_user(user_id=629e80c75d8c4c97b4d6e790d84b7b90)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:51.789597 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151de64e-91da-434c-82aa-915f96dfdc79 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:51.791491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 114/232] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:51 2022] GET /identity/v3/users/629e80c75d8c4c97b4d6e790d84b7b90 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:51.797676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1faed-1e4b-4b7f-aa00-df6fedce60bb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:51.904276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:51.908289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1faed-1e4b-4b7f-aa00-df6fedce60bb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:51.944162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b92cb-a19f-4608-91a8-c6d3dbd3e1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:51.945142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b92cb-a19f-4608-91a8-c6d3dbd3e1ae None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:51.945625 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b92cb-a19f-4608-91a8-c6d3dbd3e1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:51.946102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b92cb-a19f-4608-91a8-c6d3dbd3e1ae None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:51.947506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b92cb-a19f-4608-91a8-c6d3dbd3e1ae None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:51.949169 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b92cb-a19f-4608-91a8-c6d3dbd3e1ae None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:51.951540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 119/233] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:51 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:51.960707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151de64e-91da-434c-82aa-915f96dfdc79 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:52.082069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:52.095910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151de64e-91da-434c-82aa-915f96dfdc79 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:52.131148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cc551-eb45-480f-b339-d4de539968a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:52.131904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cc551-eb45-480f-b339-d4de539968a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:52.132190 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cc551-eb45-480f-b339-d4de539968a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:52.132466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cc551-eb45-480f-b339-d4de539968a3 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850/users/629e80c75d8c4c97b4d6e790d84b7b90/roles/0a56be0718ea495f8a241ae2447069cf` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:52.134709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cc551-eb45-480f-b339-d4de539968a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=731d452341b64364b192b87761b43850, user_id=629e80c75d8c4c97b4d6e790d84b7b90, role_id=0a56be0718ea495f8a241ae2447069cf)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:52.136200 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cc551-eb45-480f-b339-d4de539968a3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:52.156860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 115/234] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:23:51 2022] PUT /identity/v3/projects/731d452341b64364b192b87761b43850/users/629e80c75d8c4c97b4d6e790d84b7b90/roles/0a56be0718ea495f8a241ae2447069cf => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:23:52.737664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fb966-e335-489e-ad69-f8996b3cd729 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:52.738668 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fb966-e335-489e-ad69-f8996b3cd729 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:52.739190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fb966-e335-489e-ad69-f8996b3cd729 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:52.741015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 120/235] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:52.746053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab81806-dc05-472c-acad-fab06f749a22 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:52.747670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab81806-dc05-472c-acad-fab06f749a22 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:52.748292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab81806-dc05-472c-acad-fab06f749a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:52.776055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6ab81806-dc05-472c-acad-fab06f749a22 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:52.777592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6ab81806-dc05-472c-acad-fab06f749a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:52.860865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 116/236] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:52.899469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fb966-e335-489e-ad69-f8996b3cd729 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:52.941204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8069dc-12ec-4770-8b45-f8032b63dbdd None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:52.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8069dc-12ec-4770-8b45-f8032b63dbdd None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:52.942350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8069dc-12ec-4770-8b45-f8032b63dbdd None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:52.944123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 117/237] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:23:52.955938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a59e-10a7-4027-9e43-b76825170245 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:52.956965 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a59e-10a7-4027-9e43-b76825170245 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:52.957470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a59e-10a7-4027-9e43-b76825170245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:52.987171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-67c3a59e-10a7-4027-9e43-b76825170245 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:52.988804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-67c3a59e-10a7-4027-9e43-b76825170245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:53.045951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fb966-e335-489e-ad69-f8996b3cd729 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:53.068757 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 118/238] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:53.082825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb72ba5-f5c7-4ea7-86f7-6f2463df99a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:53.083783 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb72ba5-f5c7-4ea7-86f7-6f2463df99a1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:53.084263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb72ba5-f5c7-4ea7-86f7-6f2463df99a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:53.084784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb72ba5-f5c7-4ea7-86f7-6f2463df99a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:53.094673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3a59e-10a7-4027-9e43-b76825170245 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:53.101318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb72ba5-f5c7-4ea7-86f7-6f2463df99a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9cb72ba5-f5c7-4ea7-86f7-6f2463df99a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:53.107637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:53.108532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:53.109339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:53.109339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:53.109339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:53.109339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:53.109339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:53.109339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:53.109339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:53.109339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:53.109339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:53.109339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:53.109339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:23:53.109339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:53.109932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:53.112097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 121/239] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:53.122960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb72ba5-f5c7-4ea7-86f7-6f2463df99a1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:53.172234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:53.206157 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3a59e-10a7-4027-9e43-b76825170245 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:53.242527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329f9fd-9eee-44a9-a180-74972eae303c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:53.243533 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329f9fd-9eee-44a9-a180-74972eae303c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:53.244032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329f9fd-9eee-44a9-a180-74972eae303c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:53.244501 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329f9fd-9eee-44a9-a180-74972eae303c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:53.246038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329f9fd-9eee-44a9-a180-74972eae303c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:53.248493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329f9fd-9eee-44a9-a180-74972eae303c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:53.250319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 119/240] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:53.256007 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329f9fd-9eee-44a9-a180-74972eae303c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:53.304559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb72ba5-f5c7-4ea7-86f7-6f2463df99a1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:53.342282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:53.353806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbebc57-fe42-4ef3-8ee6-a29806e85c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:53.354420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbebc57-fe42-4ef3-8ee6-a29806e85c00 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:53.354710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbebc57-fe42-4ef3-8ee6-a29806e85c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:53.354988 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbebc57-fe42-4ef3-8ee6-a29806e85c00 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:53.355752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbebc57-fe42-4ef3-8ee6-a29806e85c00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:53.356890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbebc57-fe42-4ef3-8ee6-a29806e85c00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:53.358277 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbebc57-fe42-4ef3-8ee6-a29806e85c00 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:53.366775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329f9fd-9eee-44a9-a180-74972eae303c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:53.375446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 122/241] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:53.381167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbebc57-fe42-4ef3-8ee6-a29806e85c00 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:53.406359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1d3c4-4904-4072-98ca-02f31af66319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:53.407346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1d3c4-4904-4072-98ca-02f31af66319 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:53.407864 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1d3c4-4904-4072-98ca-02f31af66319 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:53.408330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1d3c4-4904-4072-98ca-02f31af66319 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:53.409214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1d3c4-4904-4072-98ca-02f31af66319 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:53.411006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1d3c4-4904-4072-98ca-02f31af66319 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:53.438099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 120/242] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:23:53 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:23:53.439866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d2478-2456-49cb-a1fd-ced019aee7ad None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:53.440358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d2478-2456-49cb-a1fd-ced019aee7ad None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:53.440834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d2478-2456-49cb-a1fd-ced019aee7ad None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:53.442466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 121/243] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:53.449473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceca06f-7823-4fcd-9a12-329d723c98c7 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:53.450137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceca06f-7823-4fcd-9a12-329d723c98c7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:53.450669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceca06f-7823-4fcd-9a12-329d723c98c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:53.490112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4ceca06f-7823-4fcd-9a12-329d723c98c7 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:53.495423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4ceca06f-7823-4fcd-9a12-329d723c98c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:53.564295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:53.573089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbebc57-fe42-4ef3-8ee6-a29806e85c00 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:53.613181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4deec6-f9ca-405e-b6d7-6e796a396645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:53.614395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4deec6-f9ca-405e-b6d7-6e796a396645 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:53.614968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4deec6-f9ca-405e-b6d7-6e796a396645 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:53.615599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4deec6-f9ca-405e-b6d7-6e796a396645 None admin] PATH_INFO: `/v3/users/23ceb1c6f77d419fba418dd479c9045a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:53.631929 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4deec6-f9ca-405e-b6d7-6e796a396645 None admin] RBAC: Authorizing `identity:get_user(user_id=23ceb1c6f77d419fba418dd479c9045a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:53.633833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4deec6-f9ca-405e-b6d7-6e796a396645 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:53.636175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 123/244] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:53 2022] GET /identity/v3/users/23ceb1c6f77d419fba418dd479c9045a => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:53.642596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4deec6-f9ca-405e-b6d7-6e796a396645 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:53.662427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 122/245] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:53.691854 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceca06f-7823-4fcd-9a12-329d723c98c7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:53.764750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:53.770456 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4deec6-f9ca-405e-b6d7-6e796a396645 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:53.821165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceca06f-7823-4fcd-9a12-329d723c98c7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:53.823321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220c1f2-a42f-46b4-b131-abf762fa4bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:53.824045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220c1f2-a42f-46b4-b131-abf762fa4bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:53.824330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220c1f2-a42f-46b4-b131-abf762fa4bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:53.825147 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220c1f2-a42f-46b4-b131-abf762fa4bc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:53.851486 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:53.867683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348abd62-f191-4660-82fc-6309828bba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:53.868435 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348abd62-f191-4660-82fc-6309828bba99 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:53.868980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348abd62-f191-4660-82fc-6309828bba99 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:53.869429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348abd62-f191-4660-82fc-6309828bba99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:53.888213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348abd62-f191-4660-82fc-6309828bba99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-348abd62-f191-4660-82fc-6309828bba99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:53.897863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:53.898973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:53.900063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:53.900063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:53.900063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:53.900063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:53.900063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:53.900063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:53.900063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:53.900063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:53.900063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:53.900063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:53.900063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:53.900063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:53.902487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 123/246] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:23:53.909163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348abd62-f191-4660-82fc-6309828bba99 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:53.910853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220c1f2-a42f-46b4-b131-abf762fa4bc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:53.912376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220c1f2-a42f-46b4-b131-abf762fa4bc2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:54.053463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348abd62-f191-4660-82fc-6309828bba99 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:54.079226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 124/247] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:54.079407 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:54.081272 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a972a-98c0-4956-9671-99152afa6c6c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:54.082102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a972a-98c0-4956-9671-99152afa6c6c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:54.082581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a972a-98c0-4956-9671-99152afa6c6c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:54.084538 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 125/248] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:54.085993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a972a-98c0-4956-9671-99152afa6c6c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:54.099415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd4daf-bc9a-448e-865a-1f0d0004225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:54.100327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd4daf-bc9a-448e-865a-1f0d0004225c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:54.100789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd4daf-bc9a-448e-865a-1f0d0004225c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:54.101244 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd4daf-bc9a-448e-865a-1f0d0004225c None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:54.102161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd4daf-bc9a-448e-865a-1f0d0004225c None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:54.102650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd4daf-bc9a-448e-865a-1f0d0004225c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:54.104293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd4daf-bc9a-448e-865a-1f0d0004225c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:54.113988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 124/249] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:54.115520 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e5583-5aba-4c58-84af-36214437d0b4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:54.116035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e5583-5aba-4c58-84af-36214437d0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:54.116553 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e5583-5aba-4c58-84af-36214437d0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:54.136803 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-477e5583-5aba-4c58-84af-36214437d0b4 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:54.139424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-477e5583-5aba-4c58-84af-36214437d0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:54.144691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:54.198130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a972a-98c0-4956-9671-99152afa6c6c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:54.216356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 125/250] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:54.217711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e5583-5aba-4c58-84af-36214437d0b4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:54.239109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cd74b-6267-4042-a180-a0ff145893cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:54.239109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd74b-6267-4042-a180-a0ff145893cc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:54.239109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd74b-6267-4042-a180-a0ff145893cc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:54.239109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd74b-6267-4042-a180-a0ff145893cc None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:54.239504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cd74b-6267-4042-a180-a0ff145893cc None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:54.239504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cd74b-6267-4042-a180-a0ff145893cc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:54.239622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 126/251] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:54 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:54.245902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cd74b-6267-4042-a180-a0ff145893cc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:54.349164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cd74b-6267-4042-a180-a0ff145893cc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:54.358069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e5583-5aba-4c58-84af-36214437d0b4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:54.366476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:54.377189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:54.386810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b27178-9d7d-4e39-a1aa-f5021620c007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:54.387585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b27178-9d7d-4e39-a1aa-f5021620c007 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:54.387956 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b27178-9d7d-4e39-a1aa-f5021620c007 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:54.388304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b27178-9d7d-4e39-a1aa-f5021620c007 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:54.389157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b27178-9d7d-4e39-a1aa-f5021620c007 None admin] RBAC: Adding query filter params (role.id=84c485f3441a4653b579e9ab39d98a95, scope.project.id=731d452341b64364b192b87761b43850, user.id=23ceb1c6f77d419fba418dd479c9045a) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:54.389535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b27178-9d7d-4e39-a1aa-f5021620c007 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:54.391046 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b27178-9d7d-4e39-a1aa-f5021620c007 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:54.400750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 127/252] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:54 2022] GET /identity/v3/role_assignments?user.id=23ceb1c6f77d419fba418dd479c9045a&scope.project.id=731d452341b64364b192b87761b43850&role.id=84c485f3441a4653b579e9ab39d98a95 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:54.400750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b27178-9d7d-4e39-a1aa-f5021620c007 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:54.420094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f5bca-0dab-48e1-8bd7-658d048371db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:54.420820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f5bca-0dab-48e1-8bd7-658d048371db None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:54.421108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f5bca-0dab-48e1-8bd7-658d048371db None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:54.421385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f5bca-0dab-48e1-8bd7-658d048371db None admin] PATH_INFO: `/v3/users/5a44a48b9d034f9788579b849e36e758` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:54.425111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f5bca-0dab-48e1-8bd7-658d048371db None admin] RBAC: Authorizing `identity:get_user(user_id=5a44a48b9d034f9788579b849e36e758)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:54.426690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f5bca-0dab-48e1-8bd7-658d048371db None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:54.428488 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 126/253] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:54 2022] GET /identity/v3/users/5a44a48b9d034f9788579b849e36e758 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:54.434574 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f5bca-0dab-48e1-8bd7-658d048371db None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:54.519303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b27178-9d7d-4e39-a1aa-f5021620c007 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:54.567798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bb7ba-a9b4-4966-9949-e5b2c02d8937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:54.568759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bb7ba-a9b4-4966-9949-e5b2c02d8937 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:54.569214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bb7ba-a9b4-4966-9949-e5b2c02d8937 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:54.569636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bb7ba-a9b4-4966-9949-e5b2c02d8937 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:54.596250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54bb7ba-a9b4-4966-9949-e5b2c02d8937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:54.601043 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f5bca-0dab-48e1-8bd7-658d048371db None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b54bb7ba-a9b4-4966-9949-e5b2c02d8937 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:54.602803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:54.603829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:54.604616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:54.604616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:54.604616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:54.604616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:54.604616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:54.604616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:54.604616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:54.604616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:54.604616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:54.604616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:54.604616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:23:54.604616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:54.614097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 128/254] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:23:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:54.620919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bb7ba-a9b4-4966-9949-e5b2c02d8937 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:54.674636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4596eac-d700-42fc-9b90-23ab08a42cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:54.676277 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4596eac-d700-42fc-9b90-23ab08a42cfd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:54.676569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4596eac-d700-42fc-9b90-23ab08a42cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:54.676851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4596eac-d700-42fc-9b90-23ab08a42cfd None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:54.678070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4596eac-d700-42fc-9b90-23ab08a42cfd None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:54.679582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4596eac-d700-42fc-9b90-23ab08a42cfd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:54.682105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 127/255] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:54 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:54.691434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:54.692154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4596eac-d700-42fc-9b90-23ab08a42cfd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:54.776698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:54.875791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bb7ba-a9b4-4966-9949-e5b2c02d8937 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:54.899238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4596eac-d700-42fc-9b90-23ab08a42cfd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:54.937023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b3604-4117-4413-9d8e-1e76d30e30fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:54.938107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b3604-4117-4413-9d8e-1e76d30e30fc None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:54.938643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b3604-4117-4413-9d8e-1e76d30e30fc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:54.939193 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b3604-4117-4413-9d8e-1e76d30e30fc None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850/users/5a44a48b9d034f9788579b849e36e758/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:54.941379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b3604-4117-4413-9d8e-1e76d30e30fc None admin] RBAC: Authorizing `identity:create_grant(project_id=731d452341b64364b192b87761b43850, user_id=5a44a48b9d034f9788579b849e36e758, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:54.943116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b3604-4117-4413-9d8e-1e76d30e30fc None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:54.969374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 128/256] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:23:54 2022] PUT /identity/v3/projects/731d452341b64364b192b87761b43850/users/5a44a48b9d034f9788579b849e36e758/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:23:54.971389 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20e8b3-4332-4020-9624-8b882946238f None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:54.973038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20e8b3-4332-4020-9624-8b882946238f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:54.973038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20e8b3-4332-4020-9624-8b882946238f None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:54.973536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 129/257] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:54.981442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0900c-874c-4581-aac8-4635b5ddd5b0 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:54.981936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0900c-874c-4581-aac8-4635b5ddd5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:54.982326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0900c-874c-4581-aac8-4635b5ddd5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:54.983842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538620a-f8c2-48a7-a622-9645030dbda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:54.984486 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538620a-f8c2-48a7-a622-9645030dbda5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:54.984768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538620a-f8c2-48a7-a622-9645030dbda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:54.985045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538620a-f8c2-48a7-a622-9645030dbda5 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:54.985782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538620a-f8c2-48a7-a622-9645030dbda5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:54.986086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538620a-f8c2-48a7-a622-9645030dbda5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:54.992842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538620a-f8c2-48a7-a622-9645030dbda5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:55.016369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-62b0900c-874c-4581-aac8-4635b5ddd5b0 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:55.017645 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-62b0900c-874c-4581-aac8-4635b5ddd5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:55.021747 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 129/258] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:55.031696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538620a-f8c2-48a7-a622-9645030dbda5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:55.150552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 130/259] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:55.195088 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0900c-874c-4581-aac8-4635b5ddd5b0 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:55.238890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538620a-f8c2-48a7-a622-9645030dbda5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:55.243755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:55.282510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65bef4-e303-4829-9892-3bf22565ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:55.283182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65bef4-e303-4829-9892-3bf22565ce1d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:55.283465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65bef4-e303-4829-9892-3bf22565ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:55.283737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65bef4-e303-4829-9892-3bf22565ce1d None admin] PATH_INFO: `/v3/users/629e80c75d8c4c97b4d6e790d84b7b90` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:55.286828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65bef4-e303-4829-9892-3bf22565ce1d None admin] RBAC: Authorizing `identity:get_user(user_id=629e80c75d8c4c97b4d6e790d84b7b90)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:55.289218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65bef4-e303-4829-9892-3bf22565ce1d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:55.292259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 130/260] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:55 2022] GET /identity/v3/users/629e80c75d8c4c97b4d6e790d84b7b90 => generated 283 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:55.299428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:55.300055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65bef4-e303-4829-9892-3bf22565ce1d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:55.426581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0900c-874c-4581-aac8-4635b5ddd5b0 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:55.489369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65bef4-e303-4829-9892-3bf22565ce1d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:55.506258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712f54c-1709-4f13-bbb1-1b9e6d119aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:55.507297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712f54c-1709-4f13-bbb1-1b9e6d119aed None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:55.507781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712f54c-1709-4f13-bbb1-1b9e6d119aed None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:55.508336 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712f54c-1709-4f13-bbb1-1b9e6d119aed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:55.526596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbb23f-6537-4037-82de-6922ef54b8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:55.527371 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbb23f-6537-4037-82de-6922ef54b8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:55.528247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbb23f-6537-4037-82de-6922ef54b8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:55.528614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbb23f-6537-4037-82de-6922ef54b8f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:55.541804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712f54c-1709-4f13-bbb1-1b9e6d119aed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9712f54c-1709-4f13-bbb1-1b9e6d119aed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:55.555820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:55.556832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:55.557619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:55.557619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:55.557619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:55.557619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:55.557619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:55.557619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:55.557619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:55.557619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:55.557619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:55.557619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:55.557619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:23:55.557619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:23:55.560902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 131/261] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:23:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:23:55.566666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712f54c-1709-4f13-bbb1-1b9e6d119aed None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:55.569838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:55.612257 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbb23f-6537-4037-82de-6922ef54b8f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:55.613989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbb23f-6537-4037-82de-6922ef54b8f8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:55.764669 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 131/262] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 07:23:55.780228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:55.782624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbb23f-6537-4037-82de-6922ef54b8f8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:55.783874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712f54c-1709-4f13-bbb1-1b9e6d119aed None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:55.821788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef448bb-ac2e-47ca-9251-b8f255e0c498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:55.822562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef448bb-ac2e-47ca-9251-b8f255e0c498 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:55.822946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef448bb-ac2e-47ca-9251-b8f255e0c498 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:55.824076 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef448bb-ac2e-47ca-9251-b8f255e0c498 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:55.824076 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef448bb-ac2e-47ca-9251-b8f255e0c498 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:55.825191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef448bb-ac2e-47ca-9251-b8f255e0c498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:55.827306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef448bb-ac2e-47ca-9251-b8f255e0c498 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:55.836193 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 132/263] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:23:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:55.842427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef448bb-ac2e-47ca-9251-b8f255e0c498 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:55.909650 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:55.962246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbb23f-6537-4037-82de-6922ef54b8f8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:55.967149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef448bb-ac2e-47ca-9251-b8f255e0c498 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:56.020040 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed3d59-1384-4255-8a33-2d045045b173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:56.020611 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed3d59-1384-4255-8a33-2d045045b173 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:56.020897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed3d59-1384-4255-8a33-2d045045b173 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:56.021178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed3d59-1384-4255-8a33-2d045045b173 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:56.022645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed3d59-1384-4255-8a33-2d045045b173 None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:56.024130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed3d59-1384-4255-8a33-2d045045b173 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:56.025818 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 132/264] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:55 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:56.033731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37d20a-a8f2-4524-9c97-65b9c2561baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:56.034394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37d20a-a8f2-4524-9c97-65b9c2561baa None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:56.034685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37d20a-a8f2-4524-9c97-65b9c2561baa None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:56.034969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37d20a-a8f2-4524-9c97-65b9c2561baa None admin] PATH_INFO: `/v3/users/629e80c75d8c4c97b4d6e790d84b7b90` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:56.036202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37d20a-a8f2-4524-9c97-65b9c2561baa None admin] RBAC: Authorizing `identity:get_user(user_id=629e80c75d8c4c97b4d6e790d84b7b90)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:56.037677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37d20a-a8f2-4524-9c97-65b9c2561baa None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:56.039334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 133/265] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:55 2022] GET /identity/v3/users/629e80c75d8c4c97b4d6e790d84b7b90 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:56.044919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed3d59-1384-4255-8a33-2d045045b173 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:56.049800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37d20a-a8f2-4524-9c97-65b9c2561baa None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:56.103177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:56.229728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed3d59-1384-4255-8a33-2d045045b173 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:56.235288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37d20a-a8f2-4524-9c97-65b9c2561baa None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:56.283668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1e63f-a9bf-4513-bc4f-3c0fbf07a258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:56.284218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1e63f-a9bf-4513-bc4f-3c0fbf07a258 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:56.284505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1e63f-a9bf-4513-bc4f-3c0fbf07a258 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:56.284790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1e63f-a9bf-4513-bc4f-3c0fbf07a258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:56.285592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1e63f-a9bf-4513-bc4f-3c0fbf07a258 None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=731d452341b64364b192b87761b43850, user.id=629e80c75d8c4c97b4d6e790d84b7b90) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:56.285911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1e63f-a9bf-4513-bc4f-3c0fbf07a258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:56.287969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1e63f-a9bf-4513-bc4f-3c0fbf07a258 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:56.294559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 134/266] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:56 2022] GET /identity/v3/role_assignments?user.id=629e80c75d8c4c97b4d6e790d84b7b90&scope.project.id=731d452341b64364b192b87761b43850&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:56.322686 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cce5c-05d5-4eae-8d64-95cfbb855084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:56.323381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cce5c-05d5-4eae-8d64-95cfbb855084 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:56.323665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cce5c-05d5-4eae-8d64-95cfbb855084 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:56.323943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cce5c-05d5-4eae-8d64-95cfbb855084 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:56.350218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:56.431941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cce5c-05d5-4eae-8d64-95cfbb855084 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:56.433430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cce5c-05d5-4eae-8d64-95cfbb855084 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:56.560163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 133/267] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:56.565704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1e63f-a9bf-4513-bc4f-3c0fbf07a258 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:56.667044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:56.677201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1e63f-a9bf-4513-bc4f-3c0fbf07a258 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:56.696629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717c68c-017d-49d4-aadd-d30bf266aa3d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:56.696925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717c68c-017d-49d4-aadd-d30bf266aa3d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:56.697197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717c68c-017d-49d4-aadd-d30bf266aa3d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:56.698741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 134/268] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:23:56.704791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d92ca4-58d0-4d21-9f0a-d6f3740b2563 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:56.705079 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d92ca4-58d0-4d21-9f0a-d6f3740b2563 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:56.705357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d92ca4-58d0-4d21-9f0a-d6f3740b2563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:56.720276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10f65e-85ce-44a0-88c5-5dae46602dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:56.720942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10f65e-85ce-44a0-88c5-5dae46602dff None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:56.721230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10f65e-85ce-44a0-88c5-5dae46602dff None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:56.722086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10f65e-85ce-44a0-88c5-5dae46602dff None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:56.727539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10f65e-85ce-44a0-88c5-5dae46602dff None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:56.729029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10f65e-85ce-44a0-88c5-5dae46602dff None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:56.731237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d1d92ca4-58d0-4d21-9f0a-d6f3740b2563 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:56.731696 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 135/269] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:56 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:56.734009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d1d92ca4-58d0-4d21-9f0a-d6f3740b2563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:56.738607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10f65e-85ce-44a0-88c5-5dae46602dff None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:56.784176 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:56.876475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 135/270] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:56.877728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10f65e-85ce-44a0-88c5-5dae46602dff None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:56.922588 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d92ca4-58d0-4d21-9f0a-d6f3740b2563 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:56.929575 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3512f98-8fe4-40c6-bd84-1569d0a943d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:56.930450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3512f98-8fe4-40c6-bd84-1569d0a943d1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:56.930948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3512f98-8fe4-40c6-bd84-1569d0a943d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:56.931456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3512f98-8fe4-40c6-bd84-1569d0a943d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:56.932430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3512f98-8fe4-40c6-bd84-1569d0a943d1 None admin] RBAC: Adding query filter params (role.id=0a56be0718ea495f8a241ae2447069cf, scope.project.id=731d452341b64364b192b87761b43850, user.id=629e80c75d8c4c97b4d6e790d84b7b90) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:56.932936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3512f98-8fe4-40c6-bd84-1569d0a943d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:56.935161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3512f98-8fe4-40c6-bd84-1569d0a943d1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:56.947959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 136/271] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:56 2022] GET /identity/v3/role_assignments?user.id=629e80c75d8c4c97b4d6e790d84b7b90&scope.project.id=731d452341b64364b192b87761b43850&role.id=0a56be0718ea495f8a241ae2447069cf => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:57.047644 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d92ca4-58d0-4d21-9f0a-d6f3740b2563 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:57.087337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50663b-16c4-4413-8c7d-4fb5e4d0b8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:57.087995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50663b-16c4-4413-8c7d-4fb5e4d0b8ff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.088276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50663b-16c4-4413-8c7d-4fb5e4d0b8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.088555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50663b-16c4-4413-8c7d-4fb5e4d0b8ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.101191 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50663b-16c4-4413-8c7d-4fb5e4d0b8ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ea50663b-16c4-4413-8c7d-4fb5e4d0b8ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:57.106678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:57.107578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:57.108768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 136/272] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:23:57.114234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3512f98-8fe4-40c6-bd84-1569d0a943d1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:57.156404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:57.257586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3512f98-8fe4-40c6-bd84-1569d0a943d1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:57.294444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f7436-96f9-42f1-864e-35f61a541884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:57.295087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f7436-96f9-42f1-864e-35f61a541884 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.295377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f7436-96f9-42f1-864e-35f61a541884 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.295654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f7436-96f9-42f1-864e-35f61a541884 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.296368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f7436-96f9-42f1-864e-35f61a541884 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:57.296675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f7436-96f9-42f1-864e-35f61a541884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:57.298070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f7436-96f9-42f1-864e-35f61a541884 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:57.310427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 137/273] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:57.316148 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50663b-16c4-4413-8c7d-4fb5e4d0b8ff None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:57.347244 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:57.424965 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50663b-16c4-4413-8c7d-4fb5e4d0b8ff None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:57.450157 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017f9e9-a662-4e21-bbb2-799e66026aef None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.450504 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017f9e9-a662-4e21-bbb2-799e66026aef None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.450724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017f9e9-a662-4e21-bbb2-799e66026aef None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.452743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 138/274] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:57.458747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f75eb-7ebb-46a6-931b-8179c56207e0 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.459109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f75eb-7ebb-46a6-931b-8179c56207e0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.459448 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f75eb-7ebb-46a6-931b-8179c56207e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.472700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40b842-688a-4552-baf9-341823c0a0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:57.472700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40b842-688a-4552-baf9-341823c0a0de None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.472700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40b842-688a-4552-baf9-341823c0a0de None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.472700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40b842-688a-4552-baf9-341823c0a0de None admin] PATH_INFO: `/v3/users/2b792184dd5243a4890da2b45171632d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.484300 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ed9f75eb-7ebb-46a6-931b-8179c56207e0 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:57.485847 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ed9f75eb-7ebb-46a6-931b-8179c56207e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:57.490477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40b842-688a-4552-baf9-341823c0a0de None admin] RBAC: Authorizing `identity:get_user(user_id=2b792184dd5243a4890da2b45171632d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:57.492685 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40b842-688a-4552-baf9-341823c0a0de None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:57.493867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 137/275] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:57 2022] GET /identity/v3/users/2b792184dd5243a4890da2b45171632d => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:57.495701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b45c44-3be2-440e-af1f-2a722b8e384b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.495701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b45c44-3be2-440e-af1f-2a722b8e384b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.495701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b45c44-3be2-440e-af1f-2a722b8e384b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.497717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 138/276] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:57.502967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dd0e4-64ae-40c1-b147-f4bb2574df82 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.503512 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dd0e4-64ae-40c1-b147-f4bb2574df82 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.503801 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dd0e4-64ae-40c1-b147-f4bb2574df82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.525388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-146dd0e4-64ae-40c1-b147-f4bb2574df82 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:57.526628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-146dd0e4-64ae-40c1-b147-f4bb2574df82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:57.575120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 139/277] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:57.575120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f75eb-7ebb-46a6-931b-8179c56207e0 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:57.594121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:57.606963 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 139/278] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:23:57.615310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dd0e4-64ae-40c1-b147-f4bb2574df82 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:57.627480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:57.695346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f75eb-7ebb-46a6-931b-8179c56207e0 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:57.715848 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dd0e4-64ae-40c1-b147-f4bb2574df82 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:57.728271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbaf69-e2f4-451f-b4e5-5069dcd708e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:57.728891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbaf69-e2f4-451f-b4e5-5069dcd708e1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.729169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbaf69-e2f4-451f-b4e5-5069dcd708e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.729447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbaf69-e2f4-451f-b4e5-5069dcd708e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.730228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbaf69-e2f4-451f-b4e5-5069dcd708e1 None admin] RBAC: Adding query filter params (role.id=d8e3b1d89cfa4e0fb43b360a31b56605, user.id=2b792184dd5243a4890da2b45171632d, scope.system=all) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:57.731074 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbaf69-e2f4-451f-b4e5-5069dcd708e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:57.732603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbaf69-e2f4-451f-b4e5-5069dcd708e1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:57.737421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 140/279] 10.222.0.40 () {56 vars in 1308 bytes} [Mon Aug 1 07:23:57 2022] GET /identity/v3/role_assignments?user.id=2b792184dd5243a4890da2b45171632d&scope.system=all&role.id=d8e3b1d89cfa4e0fb43b360a31b56605 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:57.738260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbaf69-e2f4-451f-b4e5-5069dcd708e1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:57.753918 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc643f50-8074-4937-a438-5e14615d7b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:57.754593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc643f50-8074-4937-a438-5e14615d7b17 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.754879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc643f50-8074-4937-a438-5e14615d7b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.755273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc643f50-8074-4937-a438-5e14615d7b17 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.767779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc643f50-8074-4937-a438-5e14615d7b17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-bc643f50-8074-4937-a438-5e14615d7b17 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:57.772887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:57.773833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:57.774631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 140/280] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:57.776529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a72e62-9e38-448c-8ac7-24b0dcf10f69 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.776813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a72e62-9e38-448c-8ac7-24b0dcf10f69 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.777090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a72e62-9e38-448c-8ac7-24b0dcf10f69 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.780826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 141/281] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:23:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:23:57.785699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a72e62-9e38-448c-8ac7-24b0dcf10f69 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:57.837683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:57.859696 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbaf69-e2f4-451f-b4e5-5069dcd708e1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:57.874929 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:57.897394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796c7c0-33d9-499c-95df-8321b4be182b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:57.898050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796c7c0-33d9-499c-95df-8321b4be182b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.898336 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796c7c0-33d9-499c-95df-8321b4be182b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.898618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796c7c0-33d9-499c-95df-8321b4be182b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.899839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796c7c0-33d9-499c-95df-8321b4be182b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:57.901499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796c7c0-33d9-499c-95df-8321b4be182b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:57.903853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 141/282] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:23:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:57.904686 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be2321-7eaf-4ed2-a598-e331de91ffa4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.904977 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be2321-7eaf-4ed2-a598-e331de91ffa4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.905263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be2321-7eaf-4ed2-a598-e331de91ffa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.908926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a72e62-9e38-448c-8ac7-24b0dcf10f69 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:57.926040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-96be2321-7eaf-4ed2-a598-e331de91ffa4 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:23:57.928035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-96be2321-7eaf-4ed2-a598-e331de91ffa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:23:57.947201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492a34e-56b7-4052-a912-3fe3e9f3cff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:57.947827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492a34e-56b7-4052-a912-3fe3e9f3cff5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:57.948119 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492a34e-56b7-4052-a912-3fe3e9f3cff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:57.948397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492a34e-56b7-4052-a912-3fe3e9f3cff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:57.949124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492a34e-56b7-4052-a912-3fe3e9f3cff5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:57.950003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492a34e-56b7-4052-a912-3fe3e9f3cff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:57.951439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492a34e-56b7-4052-a912-3fe3e9f3cff5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:57.960489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 142/283] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:57.961392 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492a34e-56b7-4052-a912-3fe3e9f3cff5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:58.023091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 142/284] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:23:58.023091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be2321-7eaf-4ed2-a598-e331de91ffa4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:58.089743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492a34e-56b7-4052-a912-3fe3e9f3cff5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:58.109933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:58.130641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4c8c3-b949-4005-8d88-2ba83b8b94e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:58.131355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4c8c3-b949-4005-8d88-2ba83b8b94e0 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:58.131640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4c8c3-b949-4005-8d88-2ba83b8b94e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:58.131863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be2321-7eaf-4ed2-a598-e331de91ffa4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:58.131984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4c8c3-b949-4005-8d88-2ba83b8b94e0 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:58.132582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4c8c3-b949-4005-8d88-2ba83b8b94e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:58.134958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4c8c3-b949-4005-8d88-2ba83b8b94e0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:58.147799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:58.163707 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 143/285] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:23:57 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:23:58.164522 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4c8c3-b949-4005-8d88-2ba83b8b94e0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:58.186518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e58a45-5e38-495d-af81-98d68319038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:58.187168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e58a45-5e38-495d-af81-98d68319038b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:58.187455 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e58a45-5e38-495d-af81-98d68319038b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:58.187732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e58a45-5e38-495d-af81-98d68319038b None admin] PATH_INFO: `/v3/users/23ceb1c6f77d419fba418dd479c9045a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:58.189130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e58a45-5e38-495d-af81-98d68319038b None admin] RBAC: Authorizing `identity:get_user(user_id=23ceb1c6f77d419fba418dd479c9045a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:58.190612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e58a45-5e38-495d-af81-98d68319038b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:58.192462 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 143/286] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:58 2022] GET /identity/v3/users/23ceb1c6f77d419fba418dd479c9045a => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:58.200376 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e58a45-5e38-495d-af81-98d68319038b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:58.298487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4c8c3-b949-4005-8d88-2ba83b8b94e0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:58.342011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e735f93-4e57-4277-b4e3-f7cc8d9f2eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:58.343205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e735f93-4e57-4277-b4e3-f7cc8d9f2eca None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:58.343492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e735f93-4e57-4277-b4e3-f7cc8d9f2eca None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:58.343767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e735f93-4e57-4277-b4e3-f7cc8d9f2eca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:58.356909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e58a45-5e38-495d-af81-98d68319038b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:58.358173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e735f93-4e57-4277-b4e3-f7cc8d9f2eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6e735f93-4e57-4277-b4e3-f7cc8d9f2eca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:23:58.363624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:23:58.364495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:23:58.365292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 144/287] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:23:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:23:58.371221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e735f93-4e57-4277-b4e3-f7cc8d9f2eca None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:58.396581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513af6f1-4a48-476a-b087-1d516f97c359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:58.397230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513af6f1-4a48-476a-b087-1d516f97c359 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:58.397511 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513af6f1-4a48-476a-b087-1d516f97c359 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:58.397788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513af6f1-4a48-476a-b087-1d516f97c359 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:58.399038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513af6f1-4a48-476a-b087-1d516f97c359 None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:58.400803 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513af6f1-4a48-476a-b087-1d516f97c359 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:58.403376 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 144/288] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:58 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:58.408948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513af6f1-4a48-476a-b087-1d516f97c359 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:58.432340 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:58.443233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:58.501528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e735f93-4e57-4277-b4e3-f7cc8d9f2eca None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:58.535041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513af6f1-4a48-476a-b087-1d516f97c359 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:58.544431 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067252b-3848-4b11-a047-e12be4ee4c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:58.545070 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067252b-3848-4b11-a047-e12be4ee4c54 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:58.545360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067252b-3848-4b11-a047-e12be4ee4c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:58.546076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067252b-3848-4b11-a047-e12be4ee4c54 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:58.547149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067252b-3848-4b11-a047-e12be4ee4c54 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:58.547471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067252b-3848-4b11-a047-e12be4ee4c54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:58.549258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067252b-3848-4b11-a047-e12be4ee4c54 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:58.562274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 145/289] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:23:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:23:58.567942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067252b-3848-4b11-a047-e12be4ee4c54 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:58.579051 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ef23c-d02f-4116-8e54-060517e31c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:58.579836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ef23c-d02f-4116-8e54-060517e31c5c None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:58.580123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ef23c-d02f-4116-8e54-060517e31c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:58.580406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ef23c-d02f-4116-8e54-060517e31c5c None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850/users/23ceb1c6f77d419fba418dd479c9045a/roles/84c485f3441a4653b579e9ab39d98a95` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:58.583359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ef23c-d02f-4116-8e54-060517e31c5c None admin] RBAC: Authorizing `identity:create_grant(project_id=731d452341b64364b192b87761b43850, user_id=23ceb1c6f77d419fba418dd479c9045a, role_id=84c485f3441a4653b579e9ab39d98a95)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:58.584882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ef23c-d02f-4116-8e54-060517e31c5c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:58.604223 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 145/290] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:23:58 2022] PUT /identity/v3/projects/731d452341b64364b192b87761b43850/users/23ceb1c6f77d419fba418dd479c9045a/roles/84c485f3441a4653b579e9ab39d98a95 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:23:58.692147 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067252b-3848-4b11-a047-e12be4ee4c54 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:58.715677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:58.736369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa110d71-173c-4ec7-8602-84aa2fae2de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:58.737034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa110d71-173c-4ec7-8602-84aa2fae2de4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:58.737320 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa110d71-173c-4ec7-8602-84aa2fae2de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:58.737599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa110d71-173c-4ec7-8602-84aa2fae2de4 None admin] PATH_INFO: `/v3/users/5a44a48b9d034f9788579b849e36e758` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:58.738868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa110d71-173c-4ec7-8602-84aa2fae2de4 None admin] RBAC: Authorizing `identity:get_user(user_id=5a44a48b9d034f9788579b849e36e758)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:58.740457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa110d71-173c-4ec7-8602-84aa2fae2de4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:58.742312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 146/291] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:23:58 2022] GET /identity/v3/users/5a44a48b9d034f9788579b849e36e758 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:58.747961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ef23c-d02f-4116-8e54-060517e31c5c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:58.835017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:58.861955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ef23c-d02f-4116-8e54-060517e31c5c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:58.898914 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b8565-42f2-4dbf-a39d-d20095e973b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:58.899572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b8565-42f2-4dbf-a39d-d20095e973b4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:58.899912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b8565-42f2-4dbf-a39d-d20095e973b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:58.900196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b8565-42f2-4dbf-a39d-d20095e973b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:58.986602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b8565-42f2-4dbf-a39d-d20095e973b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:58.988091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b8565-42f2-4dbf-a39d-d20095e973b4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:59.099417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:23:59.130808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 146/292] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:23:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:23:59.136359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa110d71-173c-4ec7-8602-84aa2fae2de4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:59.255533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa110d71-173c-4ec7-8602-84aa2fae2de4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:59.294363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e11cb-121f-4d69-a28d-963adf78be50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:59.295014 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e11cb-121f-4d69-a28d-963adf78be50 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:59.295325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e11cb-121f-4d69-a28d-963adf78be50 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:59.295607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e11cb-121f-4d69-a28d-963adf78be50 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:59.296797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e11cb-121f-4d69-a28d-963adf78be50 None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:59.298251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e11cb-121f-4d69-a28d-963adf78be50 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:59.300242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 147/293] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:23:59 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:23:59.306126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b8565-42f2-4dbf-a39d-d20095e973b4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:23:59.435650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b8565-42f2-4dbf-a39d-d20095e973b4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:23:59.479278 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8579d9-4b9e-4e8e-8698-c9fe274204c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:23:59.479903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8579d9-4b9e-4e8e-8698-c9fe274204c2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:23:59.480186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8579d9-4b9e-4e8e-8698-c9fe274204c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:23:59.480508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8579d9-4b9e-4e8e-8698-c9fe274204c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:23:59.481817 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8579d9-4b9e-4e8e-8698-c9fe274204c2 None admin] RBAC: Adding query filter params (role.id=d8e3b1d89cfa4e0fb43b360a31b56605, scope.project.id=731d452341b64364b192b87761b43850, user.id=5a44a48b9d034f9788579b849e36e758) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:23:59.482128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8579d9-4b9e-4e8e-8698-c9fe274204c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:23:59.483593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8579d9-4b9e-4e8e-8698-c9fe274204c2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:23:59.492419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 147/294] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:23:59 2022] GET /identity/v3/role_assignments?user.id=5a44a48b9d034f9788579b849e36e758&scope.project.id=731d452341b64364b192b87761b43850&role.id=d8e3b1d89cfa4e0fb43b360a31b56605 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:00.142696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09403834-77e8-414d-87c4-c16eec6b4353 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:00.143217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09403834-77e8-414d-87c4-c16eec6b4353 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:00.143321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09403834-77e8-414d-87c4-c16eec6b4353 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:00.144894 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 148/295] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:00.150930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de22e9-cd16-4817-a4de-b757b93162c9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:00.151356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de22e9-cd16-4817-a4de-b757b93162c9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:00.151556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de22e9-cd16-4817-a4de-b757b93162c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:00.170054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-08de22e9-cd16-4817-a4de-b757b93162c9 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:00.171283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-08de22e9-cd16-4817-a4de-b757b93162c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:00.208428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:00.243258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 148/296] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:00.265059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09403834-77e8-414d-87c4-c16eec6b4353 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:00.322089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:00.378263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09403834-77e8-414d-87c4-c16eec6b4353 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:00.418909 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ec643-0ca3-4f56-afc8-bd19fe7b0771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:00.419647 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ec643-0ca3-4f56-afc8-bd19fe7b0771 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:00.419927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ec643-0ca3-4f56-afc8-bd19fe7b0771 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:00.420281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ec643-0ca3-4f56-afc8-bd19fe7b0771 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:00.431086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ec643-0ca3-4f56-afc8-bd19fe7b0771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ca6ec643-0ca3-4f56-afc8-bd19fe7b0771 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:00.441360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:00.442255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:00.443372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 149/297] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:00.448914 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de22e9-cd16-4817-a4de-b757b93162c9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:00.559411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de22e9-cd16-4817-a4de-b757b93162c9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:00.595691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46f4dc-558b-432c-88e6-8b3a7d1e79d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:00.596318 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46f4dc-558b-432c-88e6-8b3a7d1e79d7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:00.596604 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46f4dc-558b-432c-88e6-8b3a7d1e79d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:00.596886 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46f4dc-558b-432c-88e6-8b3a7d1e79d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:00.597607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46f4dc-558b-432c-88e6-8b3a7d1e79d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:00.597916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46f4dc-558b-432c-88e6-8b3a7d1e79d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:00.599449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46f4dc-558b-432c-88e6-8b3a7d1e79d7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:00.609681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 149/298] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:24:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:00.617897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ec643-0ca3-4f56-afc8-bd19fe7b0771 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:00.626285 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daf0ac-31b2-46a9-a92e-d5ae0d74fd88 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:00.626577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daf0ac-31b2-46a9-a92e-d5ae0d74fd88 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:00.626862 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daf0ac-31b2-46a9-a92e-d5ae0d74fd88 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:00.629582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 150/299] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:24:00.635792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869646c-bb35-4524-a228-f0a3e2161e96 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:00.635792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869646c-bb35-4524-a228-f0a3e2161e96 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:00.635792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869646c-bb35-4524-a228-f0a3e2161e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:00.660939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f869646c-bb35-4524-a228-f0a3e2161e96 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:00.662574 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f869646c-bb35-4524-a228-f0a3e2161e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:00.693724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:00.762547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 151/300] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:00.772536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ec643-0ca3-4f56-afc8-bd19fe7b0771 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:00.787638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869646c-bb35-4524-a228-f0a3e2161e96 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:00.792240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:00.817056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae159863-591f-4b9b-949d-6a9ccf8c66fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:00.818175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159863-591f-4b9b-949d-6a9ccf8c66fa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:00.818670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159863-591f-4b9b-949d-6a9ccf8c66fa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:00.819210 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159863-591f-4b9b-949d-6a9ccf8c66fa None admin] PATH_INFO: `/v3/users/2b792184dd5243a4890da2b45171632d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:00.820625 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae159863-591f-4b9b-949d-6a9ccf8c66fa None admin] RBAC: Authorizing `identity:get_user(user_id=2b792184dd5243a4890da2b45171632d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:00.822335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae159863-591f-4b9b-949d-6a9ccf8c66fa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:00.825131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 150/301] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:24:00 2022] GET /identity/v3/users/2b792184dd5243a4890da2b45171632d => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:00.832065 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae159863-591f-4b9b-949d-6a9ccf8c66fa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:00.922174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869646c-bb35-4524-a228-f0a3e2161e96 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:00.964132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80ddaa-6bb8-4477-a875-0f98f5160fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:00.965415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80ddaa-6bb8-4477-a875-0f98f5160fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:00.965785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80ddaa-6bb8-4477-a875-0f98f5160fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:00.966260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80ddaa-6bb8-4477-a875-0f98f5160fb2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:00.975762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae159863-591f-4b9b-949d-6a9ccf8c66fa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:00.981098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec80ddaa-6bb8-4477-a875-0f98f5160fb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ec80ddaa-6bb8-4477-a875-0f98f5160fb2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:00.986648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:00.987578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 152/302] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:00.993494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca0223-14a2-497b-8059-26b1f4002640 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:00.994375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca0223-14a2-497b-8059-26b1f4002640 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:00.994375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca0223-14a2-497b-8059-26b1f4002640 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:00.994375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 153/303] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:00.997732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca0223-14a2-497b-8059-26b1f4002640 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:01.024964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b055cfa-700b-4a13-be95-ab78d0ed4758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:01.025662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b055cfa-700b-4a13-be95-ab78d0ed4758 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:01.025919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b055cfa-700b-4a13-be95-ab78d0ed4758 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:01.026193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b055cfa-700b-4a13-be95-ab78d0ed4758 None admin] PATH_INFO: `/v3/system/users/2b792184dd5243a4890da2b45171632d/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:01.028317 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b055cfa-700b-4a13-be95-ab78d0ed4758 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2b792184dd5243a4890da2b45171632d, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:01.029823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b055cfa-700b-4a13-be95-ab78d0ed4758 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:01.034620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:01.041803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 151/304] 10.222.0.40 () {58 vars in 1334 bytes} [Mon Aug 1 07:24:00 2022] PUT /identity/v3/system/users/2b792184dd5243a4890da2b45171632d/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:24:01.049265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ca2af-e5d9-4eea-a3f5-e5de16e6986d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:01.049773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ca2af-e5d9-4eea-a3f5-e5de16e6986d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:01.050299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ca2af-e5d9-4eea-a3f5-e5de16e6986d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:01.082503 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f02ca2af-e5d9-4eea-a3f5-e5de16e6986d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:01.084111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f02ca2af-e5d9-4eea-a3f5-e5de16e6986d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:01.141076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:01.152945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca0223-14a2-497b-8059-26b1f4002640 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:01.187107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 152/305] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:01.187107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bee94a-cda0-4c49-b842-73f0a2b80f3a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:01.187107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bee94a-cda0-4c49-b842-73f0a2b80f3a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:01.187107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bee94a-cda0-4c49-b842-73f0a2b80f3a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:01.187107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 153/306] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:01.201768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923410a-4272-4009-820b-7b871a08d576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:01.202217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6840197-cca8-435c-8b0a-e65cd8ef45e5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:01.202413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923410a-4272-4009-820b-7b871a08d576 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:01.202795 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923410a-4272-4009-820b-7b871a08d576 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:01.204153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923410a-4272-4009-820b-7b871a08d576 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:01.204153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923410a-4272-4009-820b-7b871a08d576 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:01.204306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923410a-4272-4009-820b-7b871a08d576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:01.204678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6840197-cca8-435c-8b0a-e65cd8ef45e5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:01.205192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6840197-cca8-435c-8b0a-e65cd8ef45e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:01.205729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923410a-4272-4009-820b-7b871a08d576 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:01.219769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 154/307] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:24:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:01.220572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923410a-4272-4009-820b-7b871a08d576 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:01.235351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d6840197-cca8-435c-8b0a-e65cd8ef45e5 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:01.236867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d6840197-cca8-435c-8b0a-e65cd8ef45e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:01.318902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 154/308] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:01.323335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6840197-cca8-435c-8b0a-e65cd8ef45e5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:01.334295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:01.355891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923410a-4272-4009-820b-7b871a08d576 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:01.398871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbda601-01b1-415d-8bb9-d37de8754561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:01.399710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbda601-01b1-415d-8bb9-d37de8754561 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:01.400000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbda601-01b1-415d-8bb9-d37de8754561 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:01.400279 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbda601-01b1-415d-8bb9-d37de8754561 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:01.403253 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbda601-01b1-415d-8bb9-d37de8754561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:01.403253 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbda601-01b1-415d-8bb9-d37de8754561 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:01.405840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 155/309] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:01.406601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbda601-01b1-415d-8bb9-d37de8754561 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:01.456167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6840197-cca8-435c-8b0a-e65cd8ef45e5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:01.459973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:01.502469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df084f-0aff-4d84-adf9-44d0f954a8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:01.503155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df084f-0aff-4d84-adf9-44d0f954a8ac None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:01.503445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df084f-0aff-4d84-adf9-44d0f954a8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:01.503727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df084f-0aff-4d84-adf9-44d0f954a8ac None admin] PATH_INFO: `/v3/users/770aa66c5c644fc983e5d3b3095de43c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:01.522130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df084f-0aff-4d84-adf9-44d0f954a8ac None admin] RBAC: Authorizing `identity:get_user(user_id=770aa66c5c644fc983e5d3b3095de43c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:01.524019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df084f-0aff-4d84-adf9-44d0f954a8ac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:01.526632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 155/310] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:24:01 2022] GET /identity/v3/users/770aa66c5c644fc983e5d3b3095de43c => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:01.531205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df084f-0aff-4d84-adf9-44d0f954a8ac None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:01.547385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbda601-01b1-415d-8bb9-d37de8754561 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:01.586882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971921dd-99cf-4033-adfd-48f890e46b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:01.587480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971921dd-99cf-4033-adfd-48f890e46b05 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:01.587760 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971921dd-99cf-4033-adfd-48f890e46b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:01.588033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971921dd-99cf-4033-adfd-48f890e46b05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:01.598506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971921dd-99cf-4033-adfd-48f890e46b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-971921dd-99cf-4033-adfd-48f890e46b05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:01.603871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:01.604787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:01.605610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 156/311] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:01.606271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971921dd-99cf-4033-adfd-48f890e46b05 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:01.627510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:01.647286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df084f-0aff-4d84-adf9-44d0f954a8ac None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:01.686711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30464c6-1950-4676-b59c-7c48ff9cb787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:01.687365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30464c6-1950-4676-b59c-7c48ff9cb787 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:01.688233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30464c6-1950-4676-b59c-7c48ff9cb787 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:01.688233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30464c6-1950-4676-b59c-7c48ff9cb787 None admin] PATH_INFO: `/v3/groups` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:01.688587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30464c6-1950-4676-b59c-7c48ff9cb787 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:01.690026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30464c6-1950-4676-b59c-7c48ff9cb787 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:01.719092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 156/312] 10.222.0.40 () {60 vars in 1134 bytes} [Mon Aug 1 07:24:01 2022] POST /identity/v3/groups => generated 229 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:24:01.719092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30464c6-1950-4676-b59c-7c48ff9cb787 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:01.719728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971921dd-99cf-4033-adfd-48f890e46b05 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:01.758004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:01.763814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d0eac-6e13-4aad-a68c-19ce1814d67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:01.766277 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d0eac-6e13-4aad-a68c-19ce1814d67f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:01.766838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d0eac-6e13-4aad-a68c-19ce1814d67f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:01.767428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d0eac-6e13-4aad-a68c-19ce1814d67f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:01.768538 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d0eac-6e13-4aad-a68c-19ce1814d67f None admin] RBAC: Adding query filter params (role.id=84c485f3441a4653b579e9ab39d98a95, user.id=770aa66c5c644fc983e5d3b3095de43c, scope.system=all) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:01.769637 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d0eac-6e13-4aad-a68c-19ce1814d67f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:01.772044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d0eac-6e13-4aad-a68c-19ce1814d67f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:01.780057 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 157/313] 10.222.0.40 () {56 vars in 1308 bytes} [Mon Aug 1 07:24:01 2022] GET /identity/v3/role_assignments?user.id=770aa66c5c644fc983e5d3b3095de43c&scope.system=all&role.id=84c485f3441a4653b579e9ab39d98a95 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:01.859502 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30464c6-1950-4676-b59c-7c48ff9cb787 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:01.901137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31ae44-665c-47d8-b810-9a6c8030d6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:01.901780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31ae44-665c-47d8-b810-9a6c8030d6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:01.902069 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31ae44-665c-47d8-b810-9a6c8030d6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:01.902887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31ae44-665c-47d8-b810-9a6c8030d6d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:01.903800 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31ae44-665c-47d8-b810-9a6c8030d6d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:01.904115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31ae44-665c-47d8-b810-9a6c8030d6d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:01.905547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31ae44-665c-47d8-b810-9a6c8030d6d6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:01.915388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 157/314] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:24:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:01.921201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d0eac-6e13-4aad-a68c-19ce1814d67f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:02.015178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:02.039987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d0eac-6e13-4aad-a68c-19ce1814d67f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:02.079700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f04c5-0212-475a-9b6e-1a5741b49eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:02.080697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f04c5-0212-475a-9b6e-1a5741b49eab None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:02.081203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f04c5-0212-475a-9b6e-1a5741b49eab None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:02.081683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f04c5-0212-475a-9b6e-1a5741b49eab None admin] PATH_INFO: `/v3/users/23ceb1c6f77d419fba418dd479c9045a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:02.083090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f04c5-0212-475a-9b6e-1a5741b49eab None admin] RBAC: Authorizing `identity:get_user(user_id=23ceb1c6f77d419fba418dd479c9045a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:02.084718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f04c5-0212-475a-9b6e-1a5741b49eab None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:02.086821 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 158/315] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:24:01 2022] GET /identity/v3/users/23ceb1c6f77d419fba418dd479c9045a => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:02.092273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31ae44-665c-47d8-b810-9a6c8030d6d6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:02.182628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:02.199016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31ae44-665c-47d8-b810-9a6c8030d6d6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:02.235368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca5264-3b86-456b-92f1-a2f2bd3d6298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:02.236102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca5264-3b86-456b-92f1-a2f2bd3d6298 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:02.236389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca5264-3b86-456b-92f1-a2f2bd3d6298 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:02.236668 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca5264-3b86-456b-92f1-a2f2bd3d6298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:02.317741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca5264-3b86-456b-92f1-a2f2bd3d6298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:02.319260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca5264-3b86-456b-92f1-a2f2bd3d6298 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:02.409463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:02.423514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 158/316] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:02.429200 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f04c5-0212-475a-9b6e-1a5741b49eab None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:02.534188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f04c5-0212-475a-9b6e-1a5741b49eab None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:02.570122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82388e7e-a51e-4a5c-a5df-2b0093792a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:02.570773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82388e7e-a51e-4a5c-a5df-2b0093792a65 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:02.571059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82388e7e-a51e-4a5c-a5df-2b0093792a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:02.571370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82388e7e-a51e-4a5c-a5df-2b0093792a65 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:02.572541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82388e7e-a51e-4a5c-a5df-2b0093792a65 None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:02.574042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82388e7e-a51e-4a5c-a5df-2b0093792a65 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:02.576101 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 159/317] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:02 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:02.582112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca5264-3b86-456b-92f1-a2f2bd3d6298 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:02.686490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca5264-3b86-456b-92f1-a2f2bd3d6298 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:02.722457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad75323-7ed3-4aa3-abc1-06bedcdf548d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:02.723092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad75323-7ed3-4aa3-abc1-06bedcdf548d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:02.723387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad75323-7ed3-4aa3-abc1-06bedcdf548d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:02.723667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad75323-7ed3-4aa3-abc1-06bedcdf548d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:02.724454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad75323-7ed3-4aa3-abc1-06bedcdf548d None admin] RBAC: Adding query filter params (role.id=84c485f3441a4653b579e9ab39d98a95, scope.project.id=731d452341b64364b192b87761b43850, user.id=23ceb1c6f77d419fba418dd479c9045a) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:02.724769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad75323-7ed3-4aa3-abc1-06bedcdf548d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:02.726204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad75323-7ed3-4aa3-abc1-06bedcdf548d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:02.732828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 159/318] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:02 2022] GET /identity/v3/role_assignments?user.id=23ceb1c6f77d419fba418dd479c9045a&scope.project.id=731d452341b64364b192b87761b43850&role.id=84c485f3441a4653b579e9ab39d98a95 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:03.184651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06f29c-537a-4207-be18-db92eb199c74 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:03.185022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06f29c-537a-4207-be18-db92eb199c74 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:03.185210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06f29c-537a-4207-be18-db92eb199c74 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:03.186831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 160/319] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:03.191819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6460a4-e045-46b8-a003-3948f7dec9c6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:03.192350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6460a4-e045-46b8-a003-3948f7dec9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:03.192694 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6460a4-e045-46b8-a003-3948f7dec9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:03.210888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cc6460a4-e045-46b8-a003-3948f7dec9c6 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:03.212104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cc6460a4-e045-46b8-a003-3948f7dec9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:03.257508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:03.282615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 160/320] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:03.304751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06f29c-537a-4207-be18-db92eb199c74 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:03.320271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:03.407011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06f29c-537a-4207-be18-db92eb199c74 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:03.442446 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4b921-2d89-49d3-896a-0cf59eaa6fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:03.443204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4b921-2d89-49d3-896a-0cf59eaa6fab None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:03.443406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4b921-2d89-49d3-896a-0cf59eaa6fab None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:03.443664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4b921-2d89-49d3-896a-0cf59eaa6fab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:03.454480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4b921-2d89-49d3-896a-0cf59eaa6fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-bdc4b921-2d89-49d3-896a-0cf59eaa6fab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:03.459364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:03.460260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:03.461425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 161/321] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:03.466365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6460a4-e045-46b8-a003-3948f7dec9c6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:03.565484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6460a4-e045-46b8-a003-3948f7dec9c6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:03.599210 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20a780-9bfb-4f61-93b0-4d8dbc275b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:03.599824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20a780-9bfb-4f61-93b0-4d8dbc275b02 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:03.600104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20a780-9bfb-4f61-93b0-4d8dbc275b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:03.600377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20a780-9bfb-4f61-93b0-4d8dbc275b02 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:03.601080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20a780-9bfb-4f61-93b0-4d8dbc275b02 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:03.601383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20a780-9bfb-4f61-93b0-4d8dbc275b02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:03.602750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20a780-9bfb-4f61-93b0-4d8dbc275b02 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:03.611884 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 161/322] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:24:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:03.617440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4b921-2d89-49d3-896a-0cf59eaa6fab None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:03.691166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:03.721428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4b921-2d89-49d3-896a-0cf59eaa6fab None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:03.755758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19e43a-a8af-458c-aefb-35c8a2c0ea7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:03.756401 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19e43a-a8af-458c-aefb-35c8a2c0ea7d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:03.756687 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19e43a-a8af-458c-aefb-35c8a2c0ea7d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:03.756968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19e43a-a8af-458c-aefb-35c8a2c0ea7d None admin] PATH_INFO: `/v3/users/2b792184dd5243a4890da2b45171632d` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:03.758131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19e43a-a8af-458c-aefb-35c8a2c0ea7d None admin] RBAC: Authorizing `identity:get_user(user_id=2b792184dd5243a4890da2b45171632d)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:03.759629 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19e43a-a8af-458c-aefb-35c8a2c0ea7d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:03.761796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 162/323] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:24:03 2022] GET /identity/v3/users/2b792184dd5243a4890da2b45171632d => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:03.767692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20a780-9bfb-4f61-93b0-4d8dbc275b02 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:03.811221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:03.858135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c4712-348e-4225-bcf5-42ee75024f72 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:03.858336 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c4712-348e-4225-bcf5-42ee75024f72 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:03.858608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c4712-348e-4225-bcf5-42ee75024f72 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:03.860118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 163/324] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:24:03.864874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0c883-e107-47db-b891-61bac37393fe None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:03.865169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0c883-e107-47db-b891-61bac37393fe None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:03.865454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0c883-e107-47db-b891-61bac37393fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:03.871927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20a780-9bfb-4f61-93b0-4d8dbc275b02 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:03.885489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-48f0c883-e107-47db-b891-61bac37393fe None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:03.887059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-48f0c883-e107-47db-b891-61bac37393fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:03.908167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523ee1e-7456-4240-ad9f-860bd78bd031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:03.908783 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523ee1e-7456-4240-ad9f-860bd78bd031 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:03.909072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523ee1e-7456-4240-ad9f-860bd78bd031 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:03.909344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523ee1e-7456-4240-ad9f-860bd78bd031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:03.910130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523ee1e-7456-4240-ad9f-860bd78bd031 None admin] RBAC: Adding query filter params (role.id=d8e3b1d89cfa4e0fb43b360a31b56605, user.id=2b792184dd5243a4890da2b45171632d, scope.system=all) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:03.910440 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523ee1e-7456-4240-ad9f-860bd78bd031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:03.911903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523ee1e-7456-4240-ad9f-860bd78bd031 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:03.924221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 162/325] 10.222.0.40 () {56 vars in 1308 bytes} [Mon Aug 1 07:24:03 2022] GET /identity/v3/role_assignments?user.id=2b792184dd5243a4890da2b45171632d&scope.system=all&role.id=d8e3b1d89cfa4e0fb43b360a31b56605 => generated 510 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:03.925388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f92a9-f72f-45f0-bf76-25b684b2f6e5 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:03.925678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f92a9-f72f-45f0-bf76-25b684b2f6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:03.925960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f92a9-f72f-45f0-bf76-25b684b2f6e5 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:03.927416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 163/326] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:03.932864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233004d-e098-4f81-b5ec-fd9c0deaa01c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:03.933194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233004d-e098-4f81-b5ec-fd9c0deaa01c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:03.933569 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233004d-e098-4f81-b5ec-fd9c0deaa01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:03.955306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6233004d-e098-4f81-b5ec-fd9c0deaa01c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:03.956559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6233004d-e098-4f81-b5ec-fd9c0deaa01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:03.969283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 164/327] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:03.992603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0c883-e107-47db-b891-61bac37393fe None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:04.038549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 164/328] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:04.060941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233004d-e098-4f81-b5ec-fd9c0deaa01c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:04.075036 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:04.095931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:04.113216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0c883-e107-47db-b891-61bac37393fe None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:04.149659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38723c-4fc6-4fef-ab5b-159fb5d61a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:04.150307 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38723c-4fc6-4fef-ab5b-159fb5d61a5a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:04.150593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38723c-4fc6-4fef-ab5b-159fb5d61a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:04.150871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38723c-4fc6-4fef-ab5b-159fb5d61a5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:04.152078 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38723c-4fc6-4fef-ab5b-159fb5d61a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:04.153618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38723c-4fc6-4fef-ab5b-159fb5d61a5a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:04.155249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 165/329] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:04.160776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38723c-4fc6-4fef-ab5b-159fb5d61a5a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:04.173848 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233004d-e098-4f81-b5ec-fd9c0deaa01c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:04.207431 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa87405-6a68-4597-984d-70fe6512fc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:04.208074 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa87405-6a68-4597-984d-70fe6512fc7a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:04.208360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa87405-6a68-4597-984d-70fe6512fc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:04.208645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa87405-6a68-4597-984d-70fe6512fc7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:04.218432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa87405-6a68-4597-984d-70fe6512fc7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0aa87405-6a68-4597-984d-70fe6512fc7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:04.223378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:04.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:04.225353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 165/330] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:04.230679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa87405-6a68-4597-984d-70fe6512fc7a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:04.257636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38723c-4fc6-4fef-ab5b-159fb5d61a5a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:04.290037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cb2bc-cc10-445c-b4f1-f67f2b5cd2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:04.290655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cb2bc-cc10-445c-b4f1-f67f2b5cd2a8 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:04.290945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cb2bc-cc10-445c-b4f1-f67f2b5cd2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:04.291251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cb2bc-cc10-445c-b4f1-f67f2b5cd2a8 None admin] PATH_INFO: `/v3/groups` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:04.291928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cb2bc-cc10-445c-b4f1-f67f2b5cd2a8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:04.293889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cb2bc-cc10-445c-b4f1-f67f2b5cd2a8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:04.304903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:04.315244 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 166/331] 10.222.0.40 () {60 vars in 1134 bytes} [Mon Aug 1 07:24:04 2022] POST /identity/v3/groups => generated 226 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:24:04.320192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:04.334879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa87405-6a68-4597-984d-70fe6512fc7a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:04.372663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38370d-ee0b-4519-83b5-7dccccc17297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:04.373211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38370d-ee0b-4519-83b5-7dccccc17297 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:04.373496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38370d-ee0b-4519-83b5-7dccccc17297 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:04.373784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38370d-ee0b-4519-83b5-7dccccc17297 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:04.374513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38370d-ee0b-4519-83b5-7dccccc17297 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:04.374826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38370d-ee0b-4519-83b5-7dccccc17297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:04.376267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38370d-ee0b-4519-83b5-7dccccc17297 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:04.385939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 166/332] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:24:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:04.392330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cb2bc-cc10-445c-b4f1-f67f2b5cd2a8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:04.497460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cb2bc-cc10-445c-b4f1-f67f2b5cd2a8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:04.531518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77002271-d93f-42aa-a509-9d59512b8303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:04.532172 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77002271-d93f-42aa-a509-9d59512b8303 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:04.532458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77002271-d93f-42aa-a509-9d59512b8303 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:04.532733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77002271-d93f-42aa-a509-9d59512b8303 None admin] PATH_INFO: `/v3/users/770aa66c5c644fc983e5d3b3095de43c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:04.533923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77002271-d93f-42aa-a509-9d59512b8303 None admin] RBAC: Authorizing `identity:get_user(user_id=770aa66c5c644fc983e5d3b3095de43c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:04.535400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77002271-d93f-42aa-a509-9d59512b8303 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:04.537068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 167/333] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:24:04 2022] GET /identity/v3/users/770aa66c5c644fc983e5d3b3095de43c => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:04.543303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38370d-ee0b-4519-83b5-7dccccc17297 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:04.651919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38370d-ee0b-4519-83b5-7dccccc17297 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:04.690033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244fe82-b9b4-48b1-9099-f670f63b014e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:04.690696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244fe82-b9b4-48b1-9099-f670f63b014e None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:04.690977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244fe82-b9b4-48b1-9099-f670f63b014e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:04.691287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244fe82-b9b4-48b1-9099-f670f63b014e None admin] PATH_INFO: `/v3/system/users/770aa66c5c644fc983e5d3b3095de43c/roles/84c485f3441a4653b579e9ab39d98a95` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:04.692751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244fe82-b9b4-48b1-9099-f670f63b014e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=770aa66c5c644fc983e5d3b3095de43c, role_id=84c485f3441a4653b579e9ab39d98a95)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:04.694127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244fe82-b9b4-48b1-9099-f670f63b014e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:04.702528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 167/334] 10.222.0.40 () {58 vars in 1334 bytes} [Mon Aug 1 07:24:04 2022] PUT /identity/v3/system/users/770aa66c5c644fc983e5d3b3095de43c/roles/84c485f3441a4653b579e9ab39d98a95 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:24:06.678717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1b34b-c5ac-42cc-a2dd-34aa66d5f1c9 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:06.679246 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1b34b-c5ac-42cc-a2dd-34aa66d5f1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:06.679321 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1b34b-c5ac-42cc-a2dd-34aa66d5f1c9 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:06.680918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 168/335] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:06.686516 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f650bc8-e524-4bee-aa84-2b9627fe2ce8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:06.686943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f650bc8-e524-4bee-aa84-2b9627fe2ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:06.687146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f650bc8-e524-4bee-aa84-2b9627fe2ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:06.712409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2f650bc8-e524-4bee-aa84-2b9627fe2ce8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:06.713589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2f650bc8-e524-4bee-aa84-2b9627fe2ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:06.725774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:06.799905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 168/336] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:06.828711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a65d4-61b8-4aca-b711-01aed0c7f79e None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:06.829034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a65d4-61b8-4aca-b711-01aed0c7f79e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:06.829270 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a65d4-61b8-4aca-b711-01aed0c7f79e None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:06.830618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 169/337] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:06.835970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad854c-9222-424a-a493-8d23c4040e7a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:06.836265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad854c-9222-424a-a493-8d23c4040e7a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:06.836531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad854c-9222-424a-a493-8d23c4040e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:06.846790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a65d4-61b8-4aca-b711-01aed0c7f79e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:06.866329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bfad854c-9222-424a-a493-8d23c4040e7a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:06.867702 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bfad854c-9222-424a-a493-8d23c4040e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:06.974323 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:07.003558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 169/338] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:07.044332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad854c-9222-424a-a493-8d23c4040e7a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:07.050776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a65d4-61b8-4aca-b711-01aed0c7f79e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:07.108300 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbb811-cf60-4671-92cd-ef459c197081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:07.109012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbb811-cf60-4671-92cd-ef459c197081 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:07.109305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbb811-cf60-4671-92cd-ef459c197081 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:07.109581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbb811-cf60-4671-92cd-ef459c197081 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:07.136148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbb811-cf60-4671-92cd-ef459c197081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-21fbb811-cf60-4671-92cd-ef459c197081 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:07.141353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:07.142239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:07.143045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 170/339] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:24:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:07.148438 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbb811-cf60-4671-92cd-ef459c197081 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:07.189900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:07.237523 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad854c-9222-424a-a493-8d23c4040e7a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:07.307881 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5bb09-b566-45c9-9452-c1050d5f6ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:07.309152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5bb09-b566-45c9-9452-c1050d5f6ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:07.309426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5bb09-b566-45c9-9452-c1050d5f6ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:07.309719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5bb09-b566-45c9-9452-c1050d5f6ea3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:07.333223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5bb09-b566-45c9-9452-c1050d5f6ea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fcb5bb09-b566-45c9-9452-c1050d5f6ea3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:07.338460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:07.339418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:07.340596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 170/340] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:07.348107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5bb09-b566-45c9-9452-c1050d5f6ea3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:07.379593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbb811-cf60-4671-92cd-ef459c197081 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:07.439785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:07.458449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec38cbb-413b-4121-9a6c-e7890e744f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:07.459507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec38cbb-413b-4121-9a6c-e7890e744f2a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:07.460753 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec38cbb-413b-4121-9a6c-e7890e744f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:07.461865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec38cbb-413b-4121-9a6c-e7890e744f2a None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:07.462942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec38cbb-413b-4121-9a6c-e7890e744f2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:07.464437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec38cbb-413b-4121-9a6c-e7890e744f2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:07.466494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec38cbb-413b-4121-9a6c-e7890e744f2a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:07.483482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 171/341] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:07.485252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d191d-ce13-4c1d-817e-1a1d5e320aab None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:07.487469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d191d-ce13-4c1d-817e-1a1d5e320aab None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:07.487814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d191d-ce13-4c1d-817e-1a1d5e320aab None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:07.489233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 172/342] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:07.489775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d191d-ce13-4c1d-817e-1a1d5e320aab None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:07.537701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5bb09-b566-45c9-9452-c1050d5f6ea3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:07.596509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc82e1-c87a-42e6-abbb-9f408a2a25b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:07.597164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc82e1-c87a-42e6-abbb-9f408a2a25b9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:07.597445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc82e1-c87a-42e6-abbb-9f408a2a25b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:07.597717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc82e1-c87a-42e6-abbb-9f408a2a25b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:07.598585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc82e1-c87a-42e6-abbb-9f408a2a25b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:07.598896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc82e1-c87a-42e6-abbb-9f408a2a25b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:07.600729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc82e1-c87a-42e6-abbb-9f408a2a25b9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:07.603333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:07.614758 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 171/343] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:24:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:07.618275 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d54897-c0e7-4232-b159-ed209d054142 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:07.618275 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d54897-c0e7-4232-b159-ed209d054142 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:07.618275 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d54897-c0e7-4232-b159-ed209d054142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:07.632156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d191d-ce13-4c1d-817e-1a1d5e320aab None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:07.642408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-31d54897-c0e7-4232-b159-ed209d054142 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:07.643669 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-31d54897-c0e7-4232-b159-ed209d054142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:07.680225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b149ca-fde1-4a44-b4ef-efd180fedffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:07.682649 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b149ca-fde1-4a44-b4ef-efd180fedffe None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:07.683875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b149ca-fde1-4a44-b4ef-efd180fedffe None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:07.684435 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b149ca-fde1-4a44-b4ef-efd180fedffe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:07.738097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 172/344] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:07.740229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5e430-de85-48b4-a503-35c1f3a7c8ec None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:07.740805 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5e430-de85-48b4-a503-35c1f3a7c8ec None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:07.741273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5e430-de85-48b4-a503-35c1f3a7c8ec None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:07.742976 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 173/345] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:07.743891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5e430-de85-48b4-a503-35c1f3a7c8ec None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:07.763185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:07.777567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b149ca-fde1-4a44-b4ef-efd180fedffe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:07.779183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b149ca-fde1-4a44-b4ef-efd180fedffe None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:07.862101 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5e430-de85-48b4-a503-35c1f3a7c8ec None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:07.891803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:07.894538 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 173/346] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:07.896393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3309d55-9489-4974-806e-0e6c6dba11cd None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:07.896677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3309d55-9489-4974-806e-0e6c6dba11cd None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:07.896951 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3309d55-9489-4974-806e-0e6c6dba11cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:07.904038 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb2d6c-179b-4d4e-bf8b-f9389ec7ce38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:07.905064 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb2d6c-179b-4d4e-bf8b-f9389ec7ce38 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:07.905568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb2d6c-179b-4d4e-bf8b-f9389ec7ce38 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:07.906036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb2d6c-179b-4d4e-bf8b-f9389ec7ce38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:07.916591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a3309d55-9489-4974-806e-0e6c6dba11cd None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:07.917855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a3309d55-9489-4974-806e-0e6c6dba11cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:07.983472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb2d6c-179b-4d4e-bf8b-f9389ec7ce38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:07.985399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb2d6c-179b-4d4e-bf8b-f9389ec7ce38 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:07.991787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 174/347] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:07.992799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3309d55-9489-4974-806e-0e6c6dba11cd None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:08.013156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:08.095957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 174/348] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:08.097233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb2d6c-179b-4d4e-bf8b-f9389ec7ce38 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:08.102455 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3309d55-9489-4974-806e-0e6c6dba11cd None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:08.126663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:08.141592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e55fc-fe6f-4eb9-ad72-d9222abff24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:08.142452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e55fc-fe6f-4eb9-ad72-d9222abff24f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:08.142901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e55fc-fe6f-4eb9-ad72-d9222abff24f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:08.143441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e55fc-fe6f-4eb9-ad72-d9222abff24f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:08.156317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e55fc-fe6f-4eb9-ad72-d9222abff24f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-de9e55fc-fe6f-4eb9-ad72-d9222abff24f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:08.162910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:08.164023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:08.164810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:08.164810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:08.164810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:08.164810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:08.164810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:08.164810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:08.164810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:08.164810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:08.164810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:08.164810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:08.164810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 07:24:08.164810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:08.167254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 175/349] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:08.168379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e55fc-fe6f-4eb9-ad72-d9222abff24f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:08.209195 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb2d6c-179b-4d4e-bf8b-f9389ec7ce38 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:08.246972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b540b-0885-4137-a904-394c5b4494ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:08.247593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b540b-0885-4137-a904-394c5b4494ea None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:08.247878 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b540b-0885-4137-a904-394c5b4494ea None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:08.248156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b540b-0885-4137-a904-394c5b4494ea None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:08.250621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b540b-0885-4137-a904-394c5b4494ea None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:08.251706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b540b-0885-4137-a904-394c5b4494ea None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:08.253542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 175/350] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:08 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:08.254310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b540b-0885-4137-a904-394c5b4494ea None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:08.256650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:08.281804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e55fc-fe6f-4eb9-ad72-d9222abff24f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:08.321276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ff616-939d-4fc4-b2e9-b9a225e29a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:08.322387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ff616-939d-4fc4-b2e9-b9a225e29a8a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:08.322912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ff616-939d-4fc4-b2e9-b9a225e29a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:08.323512 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ff616-939d-4fc4-b2e9-b9a225e29a8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:08.335664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ff616-939d-4fc4-b2e9-b9a225e29a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ed5ff616-939d-4fc4-b2e9-b9a225e29a8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:08.342120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:08.343022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:08.343838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:08.343838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:08.343838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:08.343838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:08.343838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:08.343838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:08.343838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:08.343838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:08.343838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:08.343838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:08.343838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:08.343838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:08.346537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 176/351] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:08.347586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ff616-939d-4fc4-b2e9-b9a225e29a8a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:08.359140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:08.366166 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b540b-0885-4137-a904-394c5b4494ea None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:08.403168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae414aa-d4ce-4aa3-98ad-b858300678c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:08.403820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae414aa-d4ce-4aa3-98ad-b858300678c6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:08.404102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae414aa-d4ce-4aa3-98ad-b858300678c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:08.404379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae414aa-d4ce-4aa3-98ad-b858300678c6 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:08.405559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae414aa-d4ce-4aa3-98ad-b858300678c6 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:08.406997 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae414aa-d4ce-4aa3-98ad-b858300678c6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:08.409792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 176/352] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:08 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:08.410186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae414aa-d4ce-4aa3-98ad-b858300678c6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:08.475998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ff616-939d-4fc4-b2e9-b9a225e29a8a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:08.514407 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742271b9-535b-4d52-a808-868e44141525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:08.515098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742271b9-535b-4d52-a808-868e44141525 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:08.515279 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742271b9-535b-4d52-a808-868e44141525 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:08.515563 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742271b9-535b-4d52-a808-868e44141525 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:08.516285 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742271b9-535b-4d52-a808-868e44141525 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:08.516602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742271b9-535b-4d52-a808-868e44141525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:08.517988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742271b9-535b-4d52-a808-868e44141525 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:08.523605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:08.533662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 177/353] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:24:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:08.535052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742271b9-535b-4d52-a808-868e44141525 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:08.543880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae414aa-d4ce-4aa3-98ad-b858300678c6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:08.592523 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d778d-f8fe-4aad-bd3f-aa059d61b26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:08.593546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d778d-f8fe-4aad-bd3f-aa059d61b26a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:08.594034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d778d-f8fe-4aad-bd3f-aa059d61b26a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:08.594319 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d778d-f8fe-4aad-bd3f-aa059d61b26a None admin] PATH_INFO: `/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:08.602537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d778d-f8fe-4aad-bd3f-aa059d61b26a None admin] RBAC: Authorizing `identity:get_group(group_id=19c6ee7e14484e4e8e8e7af3dbf306d4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:08.604026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d778d-f8fe-4aad-bd3f-aa059d61b26a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:08.606512 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 177/354] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:08 2022] GET /identity/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:08.607876 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47c2b2-0849-46b4-baeb-fed63fb00ec2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:08.607876 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47c2b2-0849-46b4-baeb-fed63fb00ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:08.610386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47c2b2-0849-46b4-baeb-fed63fb00ec2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:08.611964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 178/355] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:08.612696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47c2b2-0849-46b4-baeb-fed63fb00ec2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:08.645864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:08.664331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742271b9-535b-4d52-a808-868e44141525 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:08.717751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d5152-1d6b-4bef-9330-9e2b8a36027f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:08.718552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d5152-1d6b-4bef-9330-9e2b8a36027f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:08.718810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d5152-1d6b-4bef-9330-9e2b8a36027f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:08.719122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d5152-1d6b-4bef-9330-9e2b8a36027f None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:08.719850 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d5152-1d6b-4bef-9330-9e2b8a36027f None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:08.720158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d5152-1d6b-4bef-9330-9e2b8a36027f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:08.722224 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d5152-1d6b-4bef-9330-9e2b8a36027f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:08.730889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 178/356] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:24:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:08.732842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d5152-1d6b-4bef-9330-9e2b8a36027f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:08.750784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47c2b2-0849-46b4-baeb-fed63fb00ec2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:08.798493 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d795aa7-cc86-4294-97cb-b3b1a2d23f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:08.799324 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d795aa7-cc86-4294-97cb-b3b1a2d23f3a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:08.799565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d795aa7-cc86-4294-97cb-b3b1a2d23f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:08.799921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d795aa7-cc86-4294-97cb-b3b1a2d23f3a None admin] PATH_INFO: `/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:08.801173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d795aa7-cc86-4294-97cb-b3b1a2d23f3a None admin] RBAC: Authorizing `identity:get_group(group_id=19c6ee7e14484e4e8e8e7af3dbf306d4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:08.802708 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d795aa7-cc86-4294-97cb-b3b1a2d23f3a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:08.808422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 179/357] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:08 2022] GET /identity/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:08.808422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d795aa7-cc86-4294-97cb-b3b1a2d23f3a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:08.822177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:08.874836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d5152-1d6b-4bef-9330-9e2b8a36027f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:08.911888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c06113-37de-4551-b277-57a03dfa5572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:08.912555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c06113-37de-4551-b277-57a03dfa5572 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:08.912839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c06113-37de-4551-b277-57a03dfa5572 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:08.913112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c06113-37de-4551-b277-57a03dfa5572 None admin] PATH_INFO: `/v3/users/770aa66c5c644fc983e5d3b3095de43c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:08.914699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c06113-37de-4551-b277-57a03dfa5572 None admin] RBAC: Authorizing `identity:get_user(user_id=770aa66c5c644fc983e5d3b3095de43c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:08.916196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c06113-37de-4551-b277-57a03dfa5572 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:08.922644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 179/358] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:24:08 2022] GET /identity/v3/users/770aa66c5c644fc983e5d3b3095de43c => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:08.922644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b32c9-c44a-4106-921c-e2337733ed7f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:08.922644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b32c9-c44a-4106-921c-e2337733ed7f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:08.922644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b32c9-c44a-4106-921c-e2337733ed7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:08.924418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:08.930549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d795aa7-cc86-4294-97cb-b3b1a2d23f3a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:08.938501 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f88b32c9-c44a-4106-921c-e2337733ed7f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:08.939714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f88b32c9-c44a-4106-921c-e2337733ed7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:08.974476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5850b-2216-42bd-80d4-fba2341c24b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:08.975157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5850b-2216-42bd-80d4-fba2341c24b5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:08.975445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5850b-2216-42bd-80d4-fba2341c24b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:08.975724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5850b-2216-42bd-80d4-fba2341c24b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:08.976516 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5850b-2216-42bd-80d4-fba2341c24b5 None admin] RBAC: Adding query filter params (group.id=19c6ee7e14484e4e8e8e7af3dbf306d4, role.id=0a56be0718ea495f8a241ae2447069cf, scope.project.id=232b566033974d52b87f5e44eb52f5a2) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:08.976826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5850b-2216-42bd-80d4-fba2341c24b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:08.978245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5850b-2216-42bd-80d4-fba2341c24b5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:08.985722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 180/359] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Aug 1 07:24:08 2022] GET /identity/v3/role_assignments?group.id=19c6ee7e14484e4e8e8e7af3dbf306d4&scope.project.id=232b566033974d52b87f5e44eb52f5a2&role.id=0a56be0718ea495f8a241ae2447069cf => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:08.986377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5850b-2216-42bd-80d4-fba2341c24b5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:09.026309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 180/360] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:09.028185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b32c9-c44a-4106-921c-e2337733ed7f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:09.112379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:09.135222 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5850b-2216-42bd-80d4-fba2341c24b5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:09.163805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b32c9-c44a-4106-921c-e2337733ed7f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:09.193834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcaf2b-4b9a-4829-8c6a-386c4626e78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:09.194628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcaf2b-4b9a-4829-8c6a-386c4626e78b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:09.194915 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcaf2b-4b9a-4829-8c6a-386c4626e78b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:09.195224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcaf2b-4b9a-4829-8c6a-386c4626e78b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:09.216275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797dcdca-8170-439d-ac8d-62a3488c087a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:09.216912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dcdca-8170-439d-ac8d-62a3488c087a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:09.217196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dcdca-8170-439d-ac8d-62a3488c087a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:09.217477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dcdca-8170-439d-ac8d-62a3488c087a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:09.218281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dcdca-8170-439d-ac8d-62a3488c087a None admin] RBAC: Adding query filter params (group.id=19c6ee7e14484e4e8e8e7af3dbf306d4, role.id=d8e3b1d89cfa4e0fb43b360a31b56605, scope.project.id=232b566033974d52b87f5e44eb52f5a2) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:09.218601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dcdca-8170-439d-ac8d-62a3488c087a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:09.220137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dcdca-8170-439d-ac8d-62a3488c087a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:09.230589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 181/361] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Aug 1 07:24:09 2022] GET /identity/v3/role_assignments?group.id=19c6ee7e14484e4e8e8e7af3dbf306d4&scope.project.id=232b566033974d52b87f5e44eb52f5a2&role.id=d8e3b1d89cfa4e0fb43b360a31b56605 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:09.230589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba06913-5c79-466f-aaf3-d0354bd17075 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:09.230589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba06913-5c79-466f-aaf3-d0354bd17075 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:09.230589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba06913-5c79-466f-aaf3-d0354bd17075 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:09.230589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 182/362] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:09.230589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba06913-5c79-466f-aaf3-d0354bd17075 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:09.249626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:09.304655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dcaf2b-4b9a-4829-8c6a-386c4626e78b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:09.306155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dcaf2b-4b9a-4829-8c6a-386c4626e78b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:09.384924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba06913-5c79-466f-aaf3-d0354bd17075 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:09.422570 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:09.436684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8c287-b586-4332-83da-49a20135c8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:09.437380 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8c287-b586-4332-83da-49a20135c8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:09.437667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8c287-b586-4332-83da-49a20135c8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:09.437988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8c287-b586-4332-83da-49a20135c8f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:09.438885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8c287-b586-4332-83da-49a20135c8f7 None admin] RBAC: Adding query filter params (role.id=84c485f3441a4653b579e9ab39d98a95, user.id=770aa66c5c644fc983e5d3b3095de43c, scope.system=all) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:09.439223 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8c287-b586-4332-83da-49a20135c8f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:09.440724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8c287-b586-4332-83da-49a20135c8f7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:09.441972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 181/363] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:09.443942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcaf2b-4b9a-4829-8c6a-386c4626e78b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:09.454534 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 183/364] 10.222.0.40 () {56 vars in 1308 bytes} [Mon Aug 1 07:24:09 2022] GET /identity/v3/role_assignments?user.id=770aa66c5c644fc983e5d3b3095de43c&scope.system=all&role.id=84c485f3441a4653b579e9ab39d98a95 => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:09.455703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2e2d6-2a24-4ed2-8ad0-aeea85c97509 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:09.455997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2e2d6-2a24-4ed2-8ad0-aeea85c97509 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:09.456278 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2e2d6-2a24-4ed2-8ad0-aeea85c97509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:09.477609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3dc2e2d6-2a24-4ed2-8ad0-aeea85c97509 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:09.479220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3dc2e2d6-2a24-4ed2-8ad0-aeea85c97509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:09.555473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:09.609891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 184/365] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:09.609891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2e2d6-2a24-4ed2-8ad0-aeea85c97509 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:09.635501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcaf2b-4b9a-4829-8c6a-386c4626e78b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:09.692657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e76f7-9a4a-4fb6-bc62-9a693ce33e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:09.693318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e76f7-9a4a-4fb6-bc62-9a693ce33e39 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:09.693738 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e76f7-9a4a-4fb6-bc62-9a693ce33e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:09.694029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e76f7-9a4a-4fb6-bc62-9a693ce33e39 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:09.711696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e76f7-9a4a-4fb6-bc62-9a693ce33e39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a23e76f7-9a4a-4fb6-bc62-9a693ce33e39 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:09.719182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:09.720103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:09.720879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 182/366] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:24:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:09.722267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e76f7-9a4a-4fb6-bc62-9a693ce33e39 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:09.753199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2e2d6-2a24-4ed2-8ad0-aeea85c97509 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:09.772901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:09.805415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc90169-b4f3-4693-b4bb-3f98411c4623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:09.806233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc90169-b4f3-4693-b4bb-3f98411c4623 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:09.806518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc90169-b4f3-4693-b4bb-3f98411c4623 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:09.807173 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc90169-b4f3-4693-b4bb-3f98411c4623 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:09.809260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc90169-b4f3-4693-b4bb-3f98411c4623 None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:09.810909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc90169-b4f3-4693-b4bb-3f98411c4623 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:09.813464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 185/367] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:09 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:09.814396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc90169-b4f3-4693-b4bb-3f98411c4623 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:09.855395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e76f7-9a4a-4fb6-bc62-9a693ce33e39 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:09.875259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:09.895907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc15a0-c97d-4b1f-a970-6b97be35cd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:09.896488 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc15a0-c97d-4b1f-a970-6b97be35cd7d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:09.896747 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc15a0-c97d-4b1f-a970-6b97be35cd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:09.897022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc15a0-c97d-4b1f-a970-6b97be35cd7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:09.908945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc15a0-c97d-4b1f-a970-6b97be35cd7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-09bc15a0-c97d-4b1f-a970-6b97be35cd7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:09.914568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:09.916657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 183/368] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:09.918620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc15a0-c97d-4b1f-a970-6b97be35cd7d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:09.931076 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc90169-b4f3-4693-b4bb-3f98411c4623 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:09.971568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fccc9-e706-467a-a509-9949c1e0af6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:09.972200 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fccc9-e706-467a-a509-9949c1e0af6a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:09.972481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fccc9-e706-467a-a509-9949c1e0af6a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:09.972857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fccc9-e706-467a-a509-9949c1e0af6a None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:09.973632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fccc9-e706-467a-a509-9949c1e0af6a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:09.973938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fccc9-e706-467a-a509-9949c1e0af6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:09.975507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fccc9-e706-467a-a509-9949c1e0af6a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:09.985791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 186/369] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:09.986629 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fccc9-e706-467a-a509-9949c1e0af6a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:10.034709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:10.036547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc15a0-c97d-4b1f-a970-6b97be35cd7d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:10.071966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fc8c3-f434-4d71-9c7f-ed3a4d7b8490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:10.072887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fc8c3-f434-4d71-9c7f-ed3a4d7b8490 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:10.073315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fc8c3-f434-4d71-9c7f-ed3a4d7b8490 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:10.073928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fc8c3-f434-4d71-9c7f-ed3a4d7b8490 None admin] PATH_INFO: `/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:10.075342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fc8c3-f434-4d71-9c7f-ed3a4d7b8490 None admin] RBAC: Authorizing `identity:get_group(group_id=19c6ee7e14484e4e8e8e7af3dbf306d4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:10.076965 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fc8c3-f434-4d71-9c7f-ed3a4d7b8490 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:10.080227 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 184/370] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:09 2022] GET /identity/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:10.081167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fc8c3-f434-4d71-9c7f-ed3a4d7b8490 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:10.098090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fccc9-e706-467a-a509-9949c1e0af6a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:10.130135 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:10.135294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d9d58-68e8-47d2-8fc9-20db64a539a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:10.136365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d9d58-68e8-47d2-8fc9-20db64a539a2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:10.136727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d9d58-68e8-47d2-8fc9-20db64a539a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:10.137006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d9d58-68e8-47d2-8fc9-20db64a539a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:10.137781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d9d58-68e8-47d2-8fc9-20db64a539a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:10.138504 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d9d58-68e8-47d2-8fc9-20db64a539a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:10.140160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d9d58-68e8-47d2-8fc9-20db64a539a2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:10.149641 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 187/371] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:10.150433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d9d58-68e8-47d2-8fc9-20db64a539a2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:10.190582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fc8c3-f434-4d71-9c7f-ed3a4d7b8490 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:10.231040 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040deb8-ffa1-442e-8aaa-6c8b627c7457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:10.231970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040deb8-ffa1-442e-8aaa-6c8b627c7457 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:10.232397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040deb8-ffa1-442e-8aaa-6c8b627c7457 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:10.232816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040deb8-ffa1-442e-8aaa-6c8b627c7457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:10.264566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d9d58-68e8-47d2-8fc9-20db64a539a2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:10.267542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:10.313249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7b67c-8701-4cea-9ee7-be5e1487711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:10.313874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7b67c-8701-4cea-9ee7-be5e1487711b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:10.314155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7b67c-8701-4cea-9ee7-be5e1487711b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:10.314431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7b67c-8701-4cea-9ee7-be5e1487711b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:10.315330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7b67c-8701-4cea-9ee7-be5e1487711b None admin] RBAC: Adding query filter params (group.id=19c6ee7e14484e4e8e8e7af3dbf306d4, role.id=d8e3b1d89cfa4e0fb43b360a31b56605, scope.project.id=731d452341b64364b192b87761b43850) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:10.315625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7b67c-8701-4cea-9ee7-be5e1487711b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:10.317079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7b67c-8701-4cea-9ee7-be5e1487711b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:10.324282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040deb8-ffa1-442e-8aaa-6c8b627c7457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:10.324752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 188/372] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Aug 1 07:24:10 2022] GET /identity/v3/role_assignments?group.id=19c6ee7e14484e4e8e8e7af3dbf306d4&scope.project.id=731d452341b64364b192b87761b43850&role.id=d8e3b1d89cfa4e0fb43b360a31b56605 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:10.325684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7b67c-8701-4cea-9ee7-be5e1487711b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:10.326917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040deb8-ffa1-442e-8aaa-6c8b627c7457 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:10.399319 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:10.460851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7b67c-8701-4cea-9ee7-be5e1487711b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:10.481330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 185/373] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:10.486865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040deb8-ffa1-442e-8aaa-6c8b627c7457 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:10.502621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6246371-7ef3-44ab-bbc3-8fcac0d12344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:10.503210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6246371-7ef3-44ab-bbc3-8fcac0d12344 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:10.503495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6246371-7ef3-44ab-bbc3-8fcac0d12344 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:10.503778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6246371-7ef3-44ab-bbc3-8fcac0d12344 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:10.541963 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:10.581400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6246371-7ef3-44ab-bbc3-8fcac0d12344 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:10.582883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6246371-7ef3-44ab-bbc3-8fcac0d12344 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:10.603927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040deb8-ffa1-442e-8aaa-6c8b627c7457 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:10.644245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66122e4b-8635-4e45-b826-c3b405ea2975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:10.645220 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66122e4b-8635-4e45-b826-c3b405ea2975 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:10.645686 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66122e4b-8635-4e45-b826-c3b405ea2975 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:10.646162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66122e4b-8635-4e45-b826-c3b405ea2975 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:10.648010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66122e4b-8635-4e45-b826-c3b405ea2975 None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:10.649627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66122e4b-8635-4e45-b826-c3b405ea2975 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:10.651877 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 186/374] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:10 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:10.657339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66122e4b-8635-4e45-b826-c3b405ea2975 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:10.678404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:10.691599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 189/375] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:10.696685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6246371-7ef3-44ab-bbc3-8fcac0d12344 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:10.814076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66122e4b-8635-4e45-b826-c3b405ea2975 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:10.857043 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:10.860511 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355df8f-0537-436c-9e4f-e6c9bc4bcf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:10.861193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355df8f-0537-436c-9e4f-e6c9bc4bcf48 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:10.861656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355df8f-0537-436c-9e4f-e6c9bc4bcf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:10.862489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355df8f-0537-436c-9e4f-e6c9bc4bcf48 None admin] PATH_INFO: `/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:10.863687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355df8f-0537-436c-9e4f-e6c9bc4bcf48 None admin] RBAC: Authorizing `identity:get_group(group_id=19c6ee7e14484e4e8e8e7af3dbf306d4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:10.865131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355df8f-0537-436c-9e4f-e6c9bc4bcf48 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:10.867125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 187/376] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:10 2022] GET /identity/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:10.874655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355df8f-0537-436c-9e4f-e6c9bc4bcf48 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:10.950761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6246371-7ef3-44ab-bbc3-8fcac0d12344 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:10.987768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:11.016262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355df8f-0537-436c-9e4f-e6c9bc4bcf48 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:11.045176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e02aa-b5dd-49a8-8276-b5caf14256fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:11.045835 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e02aa-b5dd-49a8-8276-b5caf14256fb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:11.046121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e02aa-b5dd-49a8-8276-b5caf14256fb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:11.046400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e02aa-b5dd-49a8-8276-b5caf14256fb None admin] PATH_INFO: `/v3/projects/a9fa482966924d01a3dc7809fa925311` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:11.048347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e02aa-b5dd-49a8-8276-b5caf14256fb None admin] RBAC: Authorizing `identity:get_project(project_id=a9fa482966924d01a3dc7809fa925311)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:11.049806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e02aa-b5dd-49a8-8276-b5caf14256fb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:11.051525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 190/377] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:10 2022] GET /identity/v3/projects/a9fa482966924d01a3dc7809fa925311 => generated 344 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:11.058459 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e02aa-b5dd-49a8-8276-b5caf14256fb None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:11.085891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f64143-fe11-4ae6-bb52-39a9d7681d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:11.086441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f64143-fe11-4ae6-bb52-39a9d7681d50 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:11.086724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f64143-fe11-4ae6-bb52-39a9d7681d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:11.087000 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f64143-fe11-4ae6-bb52-39a9d7681d50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:11.087820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f64143-fe11-4ae6-bb52-39a9d7681d50 None admin] RBAC: Adding query filter params (group.id=19c6ee7e14484e4e8e8e7af3dbf306d4, role.id=0a56be0718ea495f8a241ae2447069cf, scope.project.id=731d452341b64364b192b87761b43850) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:11.088130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f64143-fe11-4ae6-bb52-39a9d7681d50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:11.096540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f64143-fe11-4ae6-bb52-39a9d7681d50 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:11.108560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 188/378] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Aug 1 07:24:10 2022] GET /identity/v3/role_assignments?group.id=19c6ee7e14484e4e8e8e7af3dbf306d4&scope.project.id=731d452341b64364b192b87761b43850&role.id=0a56be0718ea495f8a241ae2447069cf => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:11.196650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e02aa-b5dd-49a8-8276-b5caf14256fb None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:11.236740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c9719-e85a-4223-bd58-7539c8e1be22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:11.237397 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c9719-e85a-4223-bd58-7539c8e1be22 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:11.237748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c9719-e85a-4223-bd58-7539c8e1be22 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:11.238037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c9719-e85a-4223-bd58-7539c8e1be22 None admin] PATH_INFO: `/v3/groups/f83c5865c27048f19accb2c86c89d14f` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:11.242229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:11.246102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c9719-e85a-4223-bd58-7539c8e1be22 None admin] RBAC: Authorizing `identity:get_group(group_id=f83c5865c27048f19accb2c86c89d14f)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:11.248324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c9719-e85a-4223-bd58-7539c8e1be22 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:11.250408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 191/379] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:11 2022] GET /identity/v3/groups/f83c5865c27048f19accb2c86c89d14f => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:11.256068 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f64143-fe11-4ae6-bb52-39a9d7681d50 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:11.397149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f64143-fe11-4ae6-bb52-39a9d7681d50 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:11.435994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a69f8-a1bc-43bc-b914-5aedc57e2ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:11.436621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a69f8-a1bc-43bc-b914-5aedc57e2ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:11.436906 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a69f8-a1bc-43bc-b914-5aedc57e2ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:11.437187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a69f8-a1bc-43bc-b914-5aedc57e2ab2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:11.437978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a69f8-a1bc-43bc-b914-5aedc57e2ab2 None admin] RBAC: Adding query filter params (group.id=f83c5865c27048f19accb2c86c89d14f, role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=a9fa482966924d01a3dc7809fa925311) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:11.438293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a69f8-a1bc-43bc-b914-5aedc57e2ab2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:11.439779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a69f8-a1bc-43bc-b914-5aedc57e2ab2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:11.447451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 189/380] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Aug 1 07:24:11 2022] GET /identity/v3/role_assignments?group.id=f83c5865c27048f19accb2c86c89d14f&scope.project.id=a9fa482966924d01a3dc7809fa925311&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:11.505227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb2852-94d5-48d7-8451-559e34648bc5 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:11.505607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb2852-94d5-48d7-8451-559e34648bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:11.505953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb2852-94d5-48d7-8451-559e34648bc5 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:11.507502 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 192/381] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:11.514629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c56ed1-2010-4ddc-81df-d567f6026ae2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:11.514935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c56ed1-2010-4ddc-81df-d567f6026ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:11.515236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c56ed1-2010-4ddc-81df-d567f6026ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:11.543714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-82c56ed1-2010-4ddc-81df-d567f6026ae2 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:11.544979 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-82c56ed1-2010-4ddc-81df-d567f6026ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:11.552443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:11.638030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 190/382] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:11.672861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb2852-94d5-48d7-8451-559e34648bc5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:11.789764 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb2852-94d5-48d7-8451-559e34648bc5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:11.830442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebeaf21-ba7b-4899-a036-2b3dd9da57a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:11.831106 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebeaf21-ba7b-4899-a036-2b3dd9da57a5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:11.831396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebeaf21-ba7b-4899-a036-2b3dd9da57a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:11.831673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebeaf21-ba7b-4899-a036-2b3dd9da57a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:11.852693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebeaf21-ba7b-4899-a036-2b3dd9da57a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-6ebeaf21-ba7b-4899-a036-2b3dd9da57a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:11.858338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:11.859691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:11.860515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 193/383] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:24:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:11.867357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c56ed1-2010-4ddc-81df-d567f6026ae2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:11.930093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65e9ca-46e8-4240-b382-8f1bb235f03c None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:11.930383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65e9ca-46e8-4240-b382-8f1bb235f03c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:11.930576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65e9ca-46e8-4240-b382-8f1bb235f03c None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:11.932950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 194/384] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:11.937785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5178602-eac1-4241-aa81-65fec0a482b6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:11.938078 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5178602-eac1-4241-aa81-65fec0a482b6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:11.938362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5178602-eac1-4241-aa81-65fec0a482b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:11.992088 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f5178602-eac1-4241-aa81-65fec0a482b6 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:11.993326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f5178602-eac1-4241-aa81-65fec0a482b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:12.003587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c56ed1-2010-4ddc-81df-d567f6026ae2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:12.045811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9927b-6799-4d14-9c8b-0c4bbd2e8e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:12.046775 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9927b-6799-4d14-9c8b-0c4bbd2e8e49 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:12.047330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9927b-6799-4d14-9c8b-0c4bbd2e8e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:12.047848 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9927b-6799-4d14-9c8b-0c4bbd2e8e49 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:12.048864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9927b-6799-4d14-9c8b-0c4bbd2e8e49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:12.049361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9927b-6799-4d14-9c8b-0c4bbd2e8e49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:12.050956 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9927b-6799-4d14-9c8b-0c4bbd2e8e49 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:12.060836 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 191/385] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:12.070598 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9927b-6799-4d14-9c8b-0c4bbd2e8e49 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:12.074010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:12.125995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:12.149848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 195/386] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:12.186311 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5178602-eac1-4241-aa81-65fec0a482b6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:12.189893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9927b-6799-4d14-9c8b-0c4bbd2e8e49 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:12.229348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab772e48-6184-4ac6-8bd7-dee099f7b0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:12.230542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab772e48-6184-4ac6-8bd7-dee099f7b0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:12.231023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab772e48-6184-4ac6-8bd7-dee099f7b0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:12.231518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab772e48-6184-4ac6-8bd7-dee099f7b0b6 None admin] PATH_INFO: `/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:12.232891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab772e48-6184-4ac6-8bd7-dee099f7b0b6 None admin] RBAC: Authorizing `identity:get_group(group_id=19c6ee7e14484e4e8e8e7af3dbf306d4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:12.234511 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab772e48-6184-4ac6-8bd7-dee099f7b0b6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:12.236693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 192/387] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:12 2022] GET /identity/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:12.247402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab772e48-6184-4ac6-8bd7-dee099f7b0b6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:12.363708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5178602-eac1-4241-aa81-65fec0a482b6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:12.365645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:12.405644 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab772e48-6184-4ac6-8bd7-dee099f7b0b6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:12.446615 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c53d1-0b30-45b2-a44c-a83fb5cdb669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:12.446810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385c99a-0f38-491c-9e67-45c52306485d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:12.447216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c53d1-0b30-45b2-a44c-a83fb5cdb669 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:12.447573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c53d1-0b30-45b2-a44c-a83fb5cdb669 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:12.447858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c53d1-0b30-45b2-a44c-a83fb5cdb669 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:12.449010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385c99a-0f38-491c-9e67-45c52306485d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:12.449677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385c99a-0f38-491c-9e67-45c52306485d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:12.450166 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385c99a-0f38-491c-9e67-45c52306485d None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:12.451627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385c99a-0f38-491c-9e67-45c52306485d None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:12.459085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385c99a-0f38-491c-9e67-45c52306485d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:12.461441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 193/388] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:12 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:12.467617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385c99a-0f38-491c-9e67-45c52306485d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:12.468844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c53d1-0b30-45b2-a44c-a83fb5cdb669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-767c53d1-0b30-45b2-a44c-a83fb5cdb669 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:12.474027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:12.474909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:12.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:12.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:12.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:12.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:12.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:12.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:12.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:12.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:12.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:12.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:12.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:12.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:12.476788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 196/389] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:12.490413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c53d1-0b30-45b2-a44c-a83fb5cdb669 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:12.587805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:12.613488 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385c99a-0f38-491c-9e67-45c52306485d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:12.668909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c53d1-0b30-45b2-a44c-a83fb5cdb669 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:12.672021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c4e0d-29f8-45a6-a279-8886922a500d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:12.672867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c4e0d-29f8-45a6-a279-8886922a500d None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:12.673290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c4e0d-29f8-45a6-a279-8886922a500d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:12.673717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c4e0d-29f8-45a6-a279-8886922a500d None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2/groups/19c6ee7e14484e4e8e8e7af3dbf306d4/roles/0a56be0718ea495f8a241ae2447069cf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:12.676806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c4e0d-29f8-45a6-a279-8886922a500d None admin] RBAC: Authorizing `identity:create_grant(project_id=232b566033974d52b87f5e44eb52f5a2, group_id=19c6ee7e14484e4e8e8e7af3dbf306d4, role_id=0a56be0718ea495f8a241ae2447069cf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:12.680773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c4e0d-29f8-45a6-a279-8886922a500d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:12.701802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:12.710263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 194/390] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Aug 1 07:24:12 2022] PUT /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2/groups/19c6ee7e14484e4e8e8e7af3dbf306d4/roles/0a56be0718ea495f8a241ae2447069cf => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:24:12.717207 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14200fa-168a-4d52-bdae-c294da477cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:12.718216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14200fa-168a-4d52-bdae-c294da477cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:12.718712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14200fa-168a-4d52-bdae-c294da477cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:12.719219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14200fa-168a-4d52-bdae-c294da477cf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:12.720163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14200fa-168a-4d52-bdae-c294da477cf7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:12.720675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14200fa-168a-4d52-bdae-c294da477cf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:12.722306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14200fa-168a-4d52-bdae-c294da477cf7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:12.732817 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 197/391] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:24:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:12.738153 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c4e0d-29f8-45a6-a279-8886922a500d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:12.868316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c4e0d-29f8-45a6-a279-8886922a500d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:12.909715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7968f65-1052-480e-859d-02ce12dc46af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:12.910715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7968f65-1052-480e-859d-02ce12dc46af None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:12.911246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7968f65-1052-480e-859d-02ce12dc46af None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:12.911731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7968f65-1052-480e-859d-02ce12dc46af None admin] PATH_INFO: `/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:12.913090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7968f65-1052-480e-859d-02ce12dc46af None admin] RBAC: Authorizing `identity:get_group(group_id=19c6ee7e14484e4e8e8e7af3dbf306d4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:12.914751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7968f65-1052-480e-859d-02ce12dc46af None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:12.917046 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 195/392] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:12 2022] GET /identity/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:12.922410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14200fa-168a-4d52-bdae-c294da477cf7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:13.031586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14200fa-168a-4d52-bdae-c294da477cf7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:13.044978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:13.066995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b560e8-17c6-4e84-9f3b-670212008546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:13.067666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b560e8-17c6-4e84-9f3b-670212008546 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:13.067952 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b560e8-17c6-4e84-9f3b-670212008546 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:13.068228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b560e8-17c6-4e84-9f3b-670212008546 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:13.069398 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b560e8-17c6-4e84-9f3b-670212008546 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:13.070838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b560e8-17c6-4e84-9f3b-670212008546 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:13.072833 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 198/393] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:12 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:13.078717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7968f65-1052-480e-859d-02ce12dc46af None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:13.154562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:13.204448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7968f65-1052-480e-859d-02ce12dc46af None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:13.248342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799a4a2-3da1-496a-95fa-4d7f740f922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:13.248938 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799a4a2-3da1-496a-95fa-4d7f740f922a None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:13.249219 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799a4a2-3da1-496a-95fa-4d7f740f922a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:13.249495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799a4a2-3da1-496a-95fa-4d7f740f922a None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2/groups/19c6ee7e14484e4e8e8e7af3dbf306d4/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:13.251447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799a4a2-3da1-496a-95fa-4d7f740f922a None admin] RBAC: Authorizing `identity:create_grant(project_id=232b566033974d52b87f5e44eb52f5a2, group_id=19c6ee7e14484e4e8e8e7af3dbf306d4, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:13.252884 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799a4a2-3da1-496a-95fa-4d7f740f922a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:13.280562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 196/394] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Aug 1 07:24:13 2022] PUT /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2/groups/19c6ee7e14484e4e8e8e7af3dbf306d4/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:24:13.702301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fb2b4-8b21-4c68-9a40-0be46968cce0 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:13.702667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fb2b4-8b21-4c68-9a40-0be46968cce0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:13.702829 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fb2b4-8b21-4c68-9a40-0be46968cce0 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:13.704469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 199/395] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:13.709572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421171c5-63c4-4540-88fb-68b7b0b88864 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:13.710116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421171c5-63c4-4540-88fb-68b7b0b88864 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:13.710617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421171c5-63c4-4540-88fb-68b7b0b88864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:13.713056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df65fe-2731-4999-9d0d-b37008dd1335 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:13.713338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df65fe-2731-4999-9d0d-b37008dd1335 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:13.713612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df65fe-2731-4999-9d0d-b37008dd1335 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:13.715683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 200/396] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:24:13.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28bd2d-506c-4c1e-80fb-16f9a298fba9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:13.721202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28bd2d-506c-4c1e-80fb-16f9a298fba9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:13.721483 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28bd2d-506c-4c1e-80fb-16f9a298fba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:13.739697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6e28bd2d-506c-4c1e-80fb-16f9a298fba9 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:13.740883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6e28bd2d-506c-4c1e-80fb-16f9a298fba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:13.743954 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-421171c5-63c4-4540-88fb-68b7b0b88864 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:13.745995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-421171c5-63c4-4540-88fb-68b7b0b88864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:13.815024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 201/397] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:13.838209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28bd2d-506c-4c1e-80fb-16f9a298fba9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:13.936729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 197/398] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:13.947172 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28bd2d-506c-4c1e-80fb-16f9a298fba9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:13.971825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421171c5-63c4-4540-88fb-68b7b0b88864 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:13.979497 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:13.984669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6efc0-2293-47f8-8de8-52b9807a8add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:13.985323 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6efc0-2293-47f8-8de8-52b9807a8add None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:13.985607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6efc0-2293-47f8-8de8-52b9807a8add None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:13.986028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6efc0-2293-47f8-8de8-52b9807a8add None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:14.003494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b6efc0-2293-47f8-8de8-52b9807a8add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-d2b6efc0-2293-47f8-8de8-52b9807a8add None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:14.010010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:14.011712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:14.012600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:14.012600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:14.012600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:14.012600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:14.012600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:14.012600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:14.012600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:14.012600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:14.012600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:14.012600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:14.012600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:14.012600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:14.015534 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 202/399] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:24:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:14.024002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6efc0-2293-47f8-8de8-52b9807a8add None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:14.071752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:14.128670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421171c5-63c4-4540-88fb-68b7b0b88864 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:14.151559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6efc0-2293-47f8-8de8-52b9807a8add None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:14.174814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee0d81-1224-4386-961b-0b35645fb13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:14.175855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee0d81-1224-4386-961b-0b35645fb13b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:14.176397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee0d81-1224-4386-961b-0b35645fb13b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:14.176869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee0d81-1224-4386-961b-0b35645fb13b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:14.189357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee0d81-1224-4386-961b-0b35645fb13b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7eee0d81-1224-4386-961b-0b35645fb13b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:14.195406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:14.196482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:14.197375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:14.197375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:14.197375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:14.197375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:14.197375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:14.197375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:14.197375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:14.197375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:14.197375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:14.197375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:14.197375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:14.197375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:14.197978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6e514-5373-4ff9-b305-e9cf198a0c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:14.197978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6e514-5373-4ff9-b305-e9cf198a0c7c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:14.197978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6e514-5373-4ff9-b305-e9cf198a0c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:14.197978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6e514-5373-4ff9-b305-e9cf198a0c7c None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:14.198309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6e514-5373-4ff9-b305-e9cf198a0c7c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:14.198309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6e514-5373-4ff9-b305-e9cf198a0c7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:14.199326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6e514-5373-4ff9-b305-e9cf198a0c7c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:14.200237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 198/400] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:14.201951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dc65c-103c-48e2-af12-433412fcf9f7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:14.202441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dc65c-103c-48e2-af12-433412fcf9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:14.202933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dc65c-103c-48e2-af12-433412fcf9f7 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:14.204411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 199/401] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:14.209057 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 203/402] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:14.209786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dc65c-103c-48e2-af12-433412fcf9f7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:14.212477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134020f-5aa3-4b32-9373-4b415d16ecbc None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:14.212976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134020f-5aa3-4b32-9373-4b415d16ecbc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:14.213567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134020f-5aa3-4b32-9373-4b415d16ecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:14.234998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f134020f-5aa3-4b32-9373-4b415d16ecbc None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:14.236627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f134020f-5aa3-4b32-9373-4b415d16ecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:14.266224 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:14.317713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 204/403] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:14.324409 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134020f-5aa3-4b32-9373-4b415d16ecbc None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:14.342285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dc65c-103c-48e2-af12-433412fcf9f7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:14.364437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:14.387616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bdc10-5723-4f34-adbb-aab8b21d2d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:14.388696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bdc10-5723-4f34-adbb-aab8b21d2d6c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:14.388986 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bdc10-5723-4f34-adbb-aab8b21d2d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:14.389265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bdc10-5723-4f34-adbb-aab8b21d2d6c None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:14.389987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2bdc10-5723-4f34-adbb-aab8b21d2d6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:14.390281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2bdc10-5723-4f34-adbb-aab8b21d2d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:14.392352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2bdc10-5723-4f34-adbb-aab8b21d2d6c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:14.403055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 200/404] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:24:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:14.403900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bdc10-5723-4f34-adbb-aab8b21d2d6c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:14.464727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134020f-5aa3-4b32-9373-4b415d16ecbc None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:14.505232 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7278d-a6db-4c36-ab36-1150fe3cb0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:14.506301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7278d-a6db-4c36-ab36-1150fe3cb0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:14.506790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7278d-a6db-4c36-ab36-1150fe3cb0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:14.507315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7278d-a6db-4c36-ab36-1150fe3cb0d2 None admin] PATH_INFO: `/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:14.508997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7278d-a6db-4c36-ab36-1150fe3cb0d2 None admin] RBAC: Authorizing `identity:get_group(group_id=19c6ee7e14484e4e8e8e7af3dbf306d4)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:14.511214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7278d-a6db-4c36-ab36-1150fe3cb0d2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:14.513562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 205/405] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:14 2022] GET /identity/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:14.514595 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7278d-a6db-4c36-ab36-1150fe3cb0d2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:14.538492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bdc10-5723-4f34-adbb-aab8b21d2d6c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:14.551658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:14.583505 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d107b57-87cd-4fd2-acbc-b6579467b033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:14.584287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d107b57-87cd-4fd2-acbc-b6579467b033 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:14.584674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d107b57-87cd-4fd2-acbc-b6579467b033 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:14.584949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d107b57-87cd-4fd2-acbc-b6579467b033 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:14.599276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d107b57-87cd-4fd2-acbc-b6579467b033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8d107b57-87cd-4fd2-acbc-b6579467b033 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:14.605928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:14.606858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:14.607714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:14.607714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:14.607714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:14.607714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:14.607714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:14.607714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:14.607714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:14.607714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:14.607714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:14.607714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:14.607714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:14.607714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:14.611130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 201/406] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:14.612576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d107b57-87cd-4fd2-acbc-b6579467b033 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:14.645227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7278d-a6db-4c36-ab36-1150fe3cb0d2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:14.657462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:14.689596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f7ff1-517c-4d49-80e2-0166d980a3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:14.690791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f7ff1-517c-4d49-80e2-0166d980a3bc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:14.691335 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f7ff1-517c-4d49-80e2-0166d980a3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:14.691826 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f7ff1-517c-4d49-80e2-0166d980a3bc None admin] PATH_INFO: `/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:14.693403 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f7ff1-517c-4d49-80e2-0166d980a3bc None admin] RBAC: Authorizing `identity:get_group(group_id=19c6ee7e14484e4e8e8e7af3dbf306d4)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:14.695277 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f7ff1-517c-4d49-80e2-0166d980a3bc None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:14.697156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 206/407] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:14 2022] GET /identity/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:14.698841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f7ff1-517c-4d49-80e2-0166d980a3bc None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:14.731230 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d107b57-87cd-4fd2-acbc-b6579467b033 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:14.776029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5636d-ebfb-44cd-9dec-25662bfa3eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:14.776723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5636d-ebfb-44cd-9dec-25662bfa3eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:14.777007 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5636d-ebfb-44cd-9dec-25662bfa3eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:14.777283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5636d-ebfb-44cd-9dec-25662bfa3eb7 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:14.778893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5636d-ebfb-44cd-9dec-25662bfa3eb7 None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:14.780721 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5636d-ebfb-44cd-9dec-25662bfa3eb7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:14.788991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 202/408] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:14 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:14.790126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5636d-ebfb-44cd-9dec-25662bfa3eb7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:14.817799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:14.823663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f7ff1-517c-4d49-80e2-0166d980a3bc None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:14.865314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d9a52-c98d-49ed-a165-b55b0b5ab10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:14.866330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d9a52-c98d-49ed-a165-b55b0b5ab10b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:14.866849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d9a52-c98d-49ed-a165-b55b0b5ab10b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:14.867391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d9a52-c98d-49ed-a165-b55b0b5ab10b None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:14.868518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d9a52-c98d-49ed-a165-b55b0b5ab10b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:14.869079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d9a52-c98d-49ed-a165-b55b0b5ab10b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:14.871261 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d9a52-c98d-49ed-a165-b55b0b5ab10b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:14.881293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 207/409] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:14.882519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d9a52-c98d-49ed-a165-b55b0b5ab10b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:14.929130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5636d-ebfb-44cd-9dec-25662bfa3eb7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:14.937774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:14.984179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3d433-2200-4cf7-ad92-1d6c9880babb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:14.985173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3d433-2200-4cf7-ad92-1d6c9880babb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:14.985721 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3d433-2200-4cf7-ad92-1d6c9880babb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:14.986207 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3d433-2200-4cf7-ad92-1d6c9880babb None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:14.987625 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3d433-2200-4cf7-ad92-1d6c9880babb None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:14.992911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3d433-2200-4cf7-ad92-1d6c9880babb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:14.997763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 203/410] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:14 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:14.998607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3d433-2200-4cf7-ad92-1d6c9880babb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:15.022535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d9a52-c98d-49ed-a165-b55b0b5ab10b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:15.059461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b678f51-a164-4cba-bbc8-6dd09fc6e307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:15.060054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b678f51-a164-4cba-bbc8-6dd09fc6e307 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:15.060342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b678f51-a164-4cba-bbc8-6dd09fc6e307 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:15.060741 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b678f51-a164-4cba-bbc8-6dd09fc6e307 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850/groups/19c6ee7e14484e4e8e8e7af3dbf306d4/roles/0a56be0718ea495f8a241ae2447069cf` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:15.063046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b678f51-a164-4cba-bbc8-6dd09fc6e307 None admin] RBAC: Authorizing `identity:create_grant(project_id=731d452341b64364b192b87761b43850, group_id=19c6ee7e14484e4e8e8e7af3dbf306d4, role_id=0a56be0718ea495f8a241ae2447069cf)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:15.064621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b678f51-a164-4cba-bbc8-6dd09fc6e307 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:15.083995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 208/411] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Aug 1 07:24:14 2022] PUT /identity/v3/projects/731d452341b64364b192b87761b43850/groups/19c6ee7e14484e4e8e8e7af3dbf306d4/roles/0a56be0718ea495f8a241ae2447069cf => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:24:15.085116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b678f51-a164-4cba-bbc8-6dd09fc6e307 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:15.111370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:15.141461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3d433-2200-4cf7-ad92-1d6c9880babb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:15.180706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44121ad-6717-4b85-b35e-590c70247846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:15.181755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44121ad-6717-4b85-b35e-590c70247846 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:15.182215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44121ad-6717-4b85-b35e-590c70247846 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:15.183232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44121ad-6717-4b85-b35e-590c70247846 None admin] PATH_INFO: `/v3/groups/f83c5865c27048f19accb2c86c89d14f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:15.184899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44121ad-6717-4b85-b35e-590c70247846 None admin] RBAC: Authorizing `identity:get_group(group_id=f83c5865c27048f19accb2c86c89d14f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:15.186671 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44121ad-6717-4b85-b35e-590c70247846 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:15.193784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 204/412] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:14 2022] GET /identity/v3/groups/f83c5865c27048f19accb2c86c89d14f => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:15.195745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d1b74-9bf4-40f2-a3f2-4bdceb0d50ec None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:15.196347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d1b74-9bf4-40f2-a3f2-4bdceb0d50ec None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:15.196799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d1b74-9bf4-40f2-a3f2-4bdceb0d50ec None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:15.198577 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 205/413] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:15.200002 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d1b74-9bf4-40f2-a3f2-4bdceb0d50ec None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:15.202454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b678f51-a164-4cba-bbc8-6dd09fc6e307 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:15.245977 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078382f1-5544-4bda-a8eb-678e30a028e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:15.246418 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078382f1-5544-4bda-a8eb-678e30a028e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:15.246714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078382f1-5544-4bda-a8eb-678e30a028e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:15.246994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078382f1-5544-4bda-a8eb-678e30a028e8 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850/groups/19c6ee7e14484e4e8e8e7af3dbf306d4/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:15.249315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078382f1-5544-4bda-a8eb-678e30a028e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=731d452341b64364b192b87761b43850, group_id=19c6ee7e14484e4e8e8e7af3dbf306d4, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:15.250795 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078382f1-5544-4bda-a8eb-678e30a028e8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:15.257296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:15.269832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 209/414] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Aug 1 07:24:15 2022] PUT /identity/v3/projects/731d452341b64364b192b87761b43850/groups/19c6ee7e14484e4e8e8e7af3dbf306d4/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:24:15.271428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd017662-f98f-414c-a363-846eab1ae42e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:15.271716 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd017662-f98f-414c-a363-846eab1ae42e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:15.271995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd017662-f98f-414c-a363-846eab1ae42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:15.295682 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bd017662-f98f-414c-a363-846eab1ae42e None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:15.297072 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bd017662-f98f-414c-a363-846eab1ae42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:15.344452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d1b74-9bf4-40f2-a3f2-4bdceb0d50ec None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:15.385933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 210/415] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:15.393424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0996f52-999e-4845-9019-ce99976696b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:15.394202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0996f52-999e-4845-9019-ce99976696b5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:15.394488 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0996f52-999e-4845-9019-ce99976696b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:15.394814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0996f52-999e-4845-9019-ce99976696b5 None admin] PATH_INFO: `/v3/projects/a9fa482966924d01a3dc7809fa925311` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:15.396040 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0996f52-999e-4845-9019-ce99976696b5 None admin] RBAC: Authorizing `identity:get_project(project_id=a9fa482966924d01a3dc7809fa925311)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:15.397714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0996f52-999e-4845-9019-ce99976696b5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:15.399951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 206/416] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:15 2022] GET /identity/v3/projects/a9fa482966924d01a3dc7809fa925311 => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:15.408275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd017662-f98f-414c-a363-846eab1ae42e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:15.418666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0996f52-999e-4845-9019-ce99976696b5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:15.424264 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:15.527264 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd017662-f98f-414c-a363-846eab1ae42e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:15.564617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0996f52-999e-4845-9019-ce99976696b5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:15.573600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bfdf4-4655-436e-b76e-5c7146314855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:15.574577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bfdf4-4655-436e-b76e-5c7146314855 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:15.574772 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bfdf4-4655-436e-b76e-5c7146314855 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:15.575056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bfdf4-4655-436e-b76e-5c7146314855 None admin] PATH_INFO: `/v3/projects/a9fa482966924d01a3dc7809fa925311/groups/f83c5865c27048f19accb2c86c89d14f/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:15.577579 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654bfdf4-4655-436e-b76e-5c7146314855 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9fa482966924d01a3dc7809fa925311, group_id=f83c5865c27048f19accb2c86c89d14f, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:15.579372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:15.580268 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654bfdf4-4655-436e-b76e-5c7146314855 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:15.597971 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 211/417] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Aug 1 07:24:15 2022] PUT /identity/v3/projects/a9fa482966924d01a3dc7809fa925311/groups/f83c5865c27048f19accb2c86c89d14f/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:24:15.624677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580221f9-3f1f-4d78-9fc3-7aa0a594ca42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:15.625577 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580221f9-3f1f-4d78-9fc3-7aa0a594ca42 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:15.625968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580221f9-3f1f-4d78-9fc3-7aa0a594ca42 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:15.626360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580221f9-3f1f-4d78-9fc3-7aa0a594ca42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:15.644138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580221f9-3f1f-4d78-9fc3-7aa0a594ca42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-580221f9-3f1f-4d78-9fc3-7aa0a594ca42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:15.649772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:15.650941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:15.652058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:15.652058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:15.652058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:15.652058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:15.652058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:15.652058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:15.652058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:15.652058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:15.652058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:15.652058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:15.652058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:15.652058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:15.653934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 207/418] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:24:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:15.659154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bfdf4-4655-436e-b76e-5c7146314855 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:15.758475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:15.796145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bfdf4-4655-436e-b76e-5c7146314855 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:15.816106 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0008a6-0a50-429f-a034-bdf33b274ec6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:15.816106 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0008a6-0a50-429f-a034-bdf33b274ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:15.816635 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0008a6-0a50-429f-a034-bdf33b274ec6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:15.818327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 208/419] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:15.825681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78212e81-66ab-4be3-8703-f7ef5db159e5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:15.826114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78212e81-66ab-4be3-8703-f7ef5db159e5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:15.826494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78212e81-66ab-4be3-8703-f7ef5db159e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:15.841984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f07c01-4a8d-4ddc-ad1c-fc84dd2214e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:15.842624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f07c01-4a8d-4ddc-ad1c-fc84dd2214e6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:15.842909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f07c01-4a8d-4ddc-ad1c-fc84dd2214e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:15.843238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f07c01-4a8d-4ddc-ad1c-fc84dd2214e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:15.843960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f07c01-4a8d-4ddc-ad1c-fc84dd2214e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:15.844271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f07c01-4a8d-4ddc-ad1c-fc84dd2214e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:15.845742 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f07c01-4a8d-4ddc-ad1c-fc84dd2214e6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:15.850051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-78212e81-66ab-4be3-8703-f7ef5db159e5 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:15.851981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-78212e81-66ab-4be3-8703-f7ef5db159e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:15.859097 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 212/420] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:15.864732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f07c01-4a8d-4ddc-ad1c-fc84dd2214e6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:15.982827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 209/421] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:16.021622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78212e81-66ab-4be3-8703-f7ef5db159e5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:16.032120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f07c01-4a8d-4ddc-ad1c-fc84dd2214e6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:16.065563 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:16.068608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e17fe8-64f9-4c4c-aaa4-b799164e9512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:16.069246 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e17fe8-64f9-4c4c-aaa4-b799164e9512 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:16.069533 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e17fe8-64f9-4c4c-aaa4-b799164e9512 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:16.069853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e17fe8-64f9-4c4c-aaa4-b799164e9512 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:16.142806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e17fe8-64f9-4c4c-aaa4-b799164e9512 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:16.144271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e17fe8-64f9-4c4c-aaa4-b799164e9512 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:16.223596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:16.233935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78212e81-66ab-4be3-8703-f7ef5db159e5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:16.262521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 213/422] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:16.268070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e17fe8-64f9-4c4c-aaa4-b799164e9512 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:16.300411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:16.301144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a7218-d5b5-4907-8b36-4feacc857816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:16.301804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a7218-d5b5-4907-8b36-4feacc857816 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:16.302086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a7218-d5b5-4907-8b36-4feacc857816 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:16.302359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a7218-d5b5-4907-8b36-4feacc857816 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:16.324440 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a7218-d5b5-4907-8b36-4feacc857816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-de5a7218-d5b5-4907-8b36-4feacc857816 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:16.329348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:16.330253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:16.331050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:16.331050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:16.331050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:16.331050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:16.331050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:16.331050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:16.331050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:16.331050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:16.331050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:16.331050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:16.331050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:16.331050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:16.339511 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 210/423] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:16.349322 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a7218-d5b5-4907-8b36-4feacc857816 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:16.380536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e17fe8-64f9-4c4c-aaa4-b799164e9512 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:16.444802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081acc8e-d644-4628-bdcc-d747925f41f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:16.445449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081acc8e-d644-4628-bdcc-d747925f41f7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:16.445729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081acc8e-d644-4628-bdcc-d747925f41f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:16.446007 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081acc8e-d644-4628-bdcc-d747925f41f7 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:16.447769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081acc8e-d644-4628-bdcc-d747925f41f7 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:16.449890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081acc8e-d644-4628-bdcc-d747925f41f7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:16.451637 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 214/424] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:16 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:16.457053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081acc8e-d644-4628-bdcc-d747925f41f7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:16.522206 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a7218-d5b5-4907-8b36-4feacc857816 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:16.590036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238cc2d-dae3-4985-934d-30fde9cd9200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:16.590658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238cc2d-dae3-4985-934d-30fde9cd9200 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:16.590937 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238cc2d-dae3-4985-934d-30fde9cd9200 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:16.591235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238cc2d-dae3-4985-934d-30fde9cd9200 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:16.591949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238cc2d-dae3-4985-934d-30fde9cd9200 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:16.592254 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238cc2d-dae3-4985-934d-30fde9cd9200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:16.593636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238cc2d-dae3-4985-934d-30fde9cd9200 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:16.611489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 211/425] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:24:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:16.620645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238cc2d-dae3-4985-934d-30fde9cd9200 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:16.657658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:16.668537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:16.674339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081acc8e-d644-4628-bdcc-d747925f41f7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:16.714080 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54df11a-62b2-4e1f-a83f-1eac2ef71108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:16.715111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54df11a-62b2-4e1f-a83f-1eac2ef71108 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:16.715689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54df11a-62b2-4e1f-a83f-1eac2ef71108 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:16.716138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54df11a-62b2-4e1f-a83f-1eac2ef71108 None admin] PATH_INFO: `/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:16.717472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54df11a-62b2-4e1f-a83f-1eac2ef71108 None admin] RBAC: Authorizing `identity:get_group(group_id=19c6ee7e14484e4e8e8e7af3dbf306d4)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:16.719220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54df11a-62b2-4e1f-a83f-1eac2ef71108 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:16.721900 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 215/426] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:16 2022] GET /identity/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:16.727162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54df11a-62b2-4e1f-a83f-1eac2ef71108 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:16.796463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238cc2d-dae3-4985-934d-30fde9cd9200 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:16.860491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca0b5f-27fa-4d5c-8716-ac0ba2b330e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:16.861114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca0b5f-27fa-4d5c-8716-ac0ba2b330e3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:16.861395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca0b5f-27fa-4d5c-8716-ac0ba2b330e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:16.861670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca0b5f-27fa-4d5c-8716-ac0ba2b330e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:16.904129 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54df11a-62b2-4e1f-a83f-1eac2ef71108 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:16.962060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8df6c-7696-4e35-975f-bb1214f58d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:16.962060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8df6c-7696-4e35-975f-bb1214f58d77 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:16.962060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8df6c-7696-4e35-975f-bb1214f58d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:16.967015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8df6c-7696-4e35-975f-bb1214f58d77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:16.969849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8df6c-7696-4e35-975f-bb1214f58d77 None admin] RBAC: Adding query filter params (group.id=19c6ee7e14484e4e8e8e7af3dbf306d4, role.id=0a56be0718ea495f8a241ae2447069cf, scope.project.id=232b566033974d52b87f5e44eb52f5a2) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:16.970167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8df6c-7696-4e35-975f-bb1214f58d77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:16.973131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8df6c-7696-4e35-975f-bb1214f58d77 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:16.987982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 216/427] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Aug 1 07:24:16 2022] GET /identity/v3/role_assignments?group.id=19c6ee7e14484e4e8e8e7af3dbf306d4&scope.project.id=232b566033974d52b87f5e44eb52f5a2&role.id=0a56be0718ea495f8a241ae2447069cf => generated 611 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:16.990545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca0b5f-27fa-4d5c-8716-ac0ba2b330e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:16.992215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca0b5f-27fa-4d5c-8716-ac0ba2b330e3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:17.027864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:17.122862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 212/428] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:17.128688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8df6c-7696-4e35-975f-bb1214f58d77 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:17.180409 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:17.279554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8df6c-7696-4e35-975f-bb1214f58d77 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:17.324074 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c1e20-ff52-41ec-ac7b-cf64cc139c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:17.324732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c1e20-ff52-41ec-ac7b-cf64cc139c66 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:17.325020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c1e20-ff52-41ec-ac7b-cf64cc139c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:17.325303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c1e20-ff52-41ec-ac7b-cf64cc139c66 None admin] PATH_INFO: `/v3/projects/232b566033974d52b87f5e44eb52f5a2` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:17.328508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c1e20-ff52-41ec-ac7b-cf64cc139c66 None admin] RBAC: Authorizing `identity:get_project(project_id=232b566033974d52b87f5e44eb52f5a2)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:17.329965 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c1e20-ff52-41ec-ac7b-cf64cc139c66 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:17.332022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 217/429] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:17 2022] GET /identity/v3/projects/232b566033974d52b87f5e44eb52f5a2 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:17.341289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca0b5f-27fa-4d5c-8716-ac0ba2b330e3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:17.448642 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca0b5f-27fa-4d5c-8716-ac0ba2b330e3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:17.473342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:17.489710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1299963-a820-417f-993b-0467d61354cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:17.490353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1299963-a820-417f-993b-0467d61354cc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:17.490634 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1299963-a820-417f-993b-0467d61354cc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:17.490908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1299963-a820-417f-993b-0467d61354cc None admin] PATH_INFO: `/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:17.492094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1299963-a820-417f-993b-0467d61354cc None admin] RBAC: Authorizing `identity:get_group(group_id=19c6ee7e14484e4e8e8e7af3dbf306d4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:17.493531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1299963-a820-417f-993b-0467d61354cc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:17.496055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 213/430] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:17 2022] GET /identity/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:17.501701 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c1e20-ff52-41ec-ac7b-cf64cc139c66 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:17.505205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dfdff-7ac7-4809-8b01-ac99136e4418 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:17.505496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dfdff-7ac7-4809-8b01-ac99136e4418 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:17.506661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dfdff-7ac7-4809-8b01-ac99136e4418 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:17.507364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 214/431] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:24:17.513798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4733d7-ac07-4bc4-887e-4d1798640fe8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:17.514253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4733d7-ac07-4bc4-887e-4d1798640fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:17.514673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4733d7-ac07-4bc4-887e-4d1798640fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:17.538955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9e4733d7-ac07-4bc4-887e-4d1798640fe8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:17.540120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9e4733d7-ac07-4bc4-887e-4d1798640fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:17.623691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:17.626623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c1e20-ff52-41ec-ac7b-cf64cc139c66 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:17.639691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 215/432] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:17.662274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4733d7-ac07-4bc4-887e-4d1798640fe8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:17.671822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d769f-525a-4556-ac98-7dc786ae7fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:17.672764 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d769f-525a-4556-ac98-7dc786ae7fef None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:17.673689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d769f-525a-4556-ac98-7dc786ae7fef None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:17.674175 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d769f-525a-4556-ac98-7dc786ae7fef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:17.675631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d769f-525a-4556-ac98-7dc786ae7fef None admin] RBAC: Adding query filter params (group.id=19c6ee7e14484e4e8e8e7af3dbf306d4, role.id=d8e3b1d89cfa4e0fb43b360a31b56605, scope.project.id=232b566033974d52b87f5e44eb52f5a2) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:17.676409 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d769f-525a-4556-ac98-7dc786ae7fef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:17.678354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d769f-525a-4556-ac98-7dc786ae7fef None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:17.695958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 218/433] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Aug 1 07:24:17 2022] GET /identity/v3/role_assignments?group.id=19c6ee7e14484e4e8e8e7af3dbf306d4&scope.project.id=232b566033974d52b87f5e44eb52f5a2&role.id=d8e3b1d89cfa4e0fb43b360a31b56605 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:17.709141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de35f1-d591-4550-a206-c68d3fbbdd20 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:17.709469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de35f1-d591-4550-a206-c68d3fbbdd20 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:17.709707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de35f1-d591-4550-a206-c68d3fbbdd20 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:17.711953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 219/434] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:24:17.716952 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a65ef-8f2b-437f-b728-1fbad1ee2359 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:17.717243 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a65ef-8f2b-437f-b728-1fbad1ee2359 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:17.717507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a65ef-8f2b-437f-b728-1fbad1ee2359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:17.738174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-eb7a65ef-8f2b-437f-b728-1fbad1ee2359 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:17.739660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-eb7a65ef-8f2b-437f-b728-1fbad1ee2359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:17.789882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4733d7-ac07-4bc4-887e-4d1798640fe8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:17.814060 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:17.833066 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a479f14-03d5-41cc-a556-c11812c4fdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:17.833732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a479f14-03d5-41cc-a556-c11812c4fdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:17.834076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a479f14-03d5-41cc-a556-c11812c4fdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:17.834281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a479f14-03d5-41cc-a556-c11812c4fdc8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:17.847113 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 220/435] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:17.854080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a479f14-03d5-41cc-a556-c11812c4fdc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1a479f14-03d5-41cc-a556-c11812c4fdc8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:17.859926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:17.860992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:17.861780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:17.861780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:17.861780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:17.861780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:17.861780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:17.861780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:17.861780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:17.861780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:17.861780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:17.861780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:17.861780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:17.861780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:17.863717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 216/436] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:24:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:17.872708 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a479f14-03d5-41cc-a556-c11812c4fdc8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:17.872870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a65ef-8f2b-437f-b728-1fbad1ee2359 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:17.996632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:18.005794 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a65ef-8f2b-437f-b728-1fbad1ee2359 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:18.014897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a479f14-03d5-41cc-a556-c11812c4fdc8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:18.053575 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc709191-57ad-4005-b29b-be3e6595addd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:18.054147 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc709191-57ad-4005-b29b-be3e6595addd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:18.054451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc709191-57ad-4005-b29b-be3e6595addd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:18.054750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc709191-57ad-4005-b29b-be3e6595addd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:18.064157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ff3f6c-8777-4c83-b1a9-b25ac2d5b3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:18.064800 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff3f6c-8777-4c83-b1a9-b25ac2d5b3ae None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:18.065084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff3f6c-8777-4c83-b1a9-b25ac2d5b3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:18.065360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff3f6c-8777-4c83-b1a9-b25ac2d5b3ae None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:18.066089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ff3f6c-8777-4c83-b1a9-b25ac2d5b3ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:18.066400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ff3f6c-8777-4c83-b1a9-b25ac2d5b3ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:18.068312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ff3f6c-8777-4c83-b1a9-b25ac2d5b3ae None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:18.071275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc709191-57ad-4005-b29b-be3e6595addd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-cc709191-57ad-4005-b29b-be3e6595addd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:18.077215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:18.078125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:18.078929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:18.078929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:18.078929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:18.078929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:18.078929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:18.078929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:18.078929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:18.078929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:18.078929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:18.078929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:18.078929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 07:24:18.078929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:18.079586 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 217/437] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:18.079670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 221/438] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:18.084381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ff3f6c-8777-4c83-b1a9-b25ac2d5b3ae None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:18.084503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc709191-57ad-4005-b29b-be3e6595addd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:18.135106 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:18.212835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ff3f6c-8777-4c83-b1a9-b25ac2d5b3ae None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:18.215048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc709191-57ad-4005-b29b-be3e6595addd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:18.254536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5049c2-b1f9-477a-9570-5059a8638783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:18.255181 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5049c2-b1f9-477a-9570-5059a8638783 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:18.255428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf213c-bc0b-49f7-a63d-628651b1b24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:18.255567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5049c2-b1f9-477a-9570-5059a8638783 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:18.255731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5049c2-b1f9-477a-9570-5059a8638783 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:18.256053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf213c-bc0b-49f7-a63d-628651b1b24a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:18.256340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf213c-bc0b-49f7-a63d-628651b1b24a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:18.256440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5049c2-b1f9-477a-9570-5059a8638783 None admin] RBAC: Adding query filter params (name=member) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:18.256619 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf213c-bc0b-49f7-a63d-628651b1b24a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:18.256789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5049c2-b1f9-477a-9570-5059a8638783 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:18.258125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5049c2-b1f9-477a-9570-5059a8638783 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:18.270895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 222/439] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:24:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:18.272235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf587ce-11b1-45ac-b601-0bd7dc5291bf None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:18.272864 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf587ce-11b1-45ac-b601-0bd7dc5291bf None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:18.273150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf587ce-11b1-45ac-b601-0bd7dc5291bf None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:18.275056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 223/440] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:18.276615 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf587ce-11b1-45ac-b601-0bd7dc5291bf None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:18.298723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:18.344129 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf213c-bc0b-49f7-a63d-628651b1b24a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:18.345518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf213c-bc0b-49f7-a63d-628651b1b24a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:18.387859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:18.391150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf587ce-11b1-45ac-b601-0bd7dc5291bf None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:18.424120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66215f-ea29-4642-9b26-3d63d18d3393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:18.424728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66215f-ea29-4642-9b26-3d63d18d3393 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:18.425010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66215f-ea29-4642-9b26-3d63d18d3393 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:18.425290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66215f-ea29-4642-9b26-3d63d18d3393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:18.449373 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 218/441] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:18.450779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b53057-b693-41e8-bffa-479da37cb8eb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:18.451085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b53057-b693-41e8-bffa-479da37cb8eb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:18.451378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b53057-b693-41e8-bffa-479da37cb8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:18.469350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-60b53057-b693-41e8-bffa-479da37cb8eb None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:18.470551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-60b53057-b693-41e8-bffa-479da37cb8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:18.494513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66215f-ea29-4642-9b26-3d63d18d3393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:18.497127 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66215f-ea29-4642-9b26-3d63d18d3393 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:18.513806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:18.536831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 219/442] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:18.537488 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b53057-b693-41e8-bffa-479da37cb8eb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:18.606351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:18.611274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 224/443] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 07:24:18.612272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66215f-ea29-4642-9b26-3d63d18d3393 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:18.640006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b53057-b693-41e8-bffa-479da37cb8eb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:18.673802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d16f3-78e2-484e-b2bc-03fdc3db0c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:18.674429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d16f3-78e2-484e-b2bc-03fdc3db0c4a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:18.674715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d16f3-78e2-484e-b2bc-03fdc3db0c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:18.674988 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d16f3-78e2-484e-b2bc-03fdc3db0c4a None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:18.676153 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d16f3-78e2-484e-b2bc-03fdc3db0c4a None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:18.677642 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d16f3-78e2-484e-b2bc-03fdc3db0c4a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:18.679319 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 220/444] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:18 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:18.680064 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d16f3-78e2-484e-b2bc-03fdc3db0c4a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:18.717431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66215f-ea29-4642-9b26-3d63d18d3393 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:18.734146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:18.752026 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2fd82e-9659-4a14-a715-3f3d8b067f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:18.752669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fd82e-9659-4a14-a715-3f3d8b067f85 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:18.752956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fd82e-9659-4a14-a715-3f3d8b067f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:18.753236 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fd82e-9659-4a14-a715-3f3d8b067f85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:18.763809 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2fd82e-9659-4a14-a715-3f3d8b067f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-8b2fd82e-9659-4a14-a715-3f3d8b067f85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:18.768763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:18.769656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:18.770437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 225/445] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:18.771304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2fd82e-9659-4a14-a715-3f3d8b067f85 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:18.784071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d16f3-78e2-484e-b2bc-03fdc3db0c4a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:18.818596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce4630-a489-4a9d-b552-2874807edb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:18.819519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce4630-a489-4a9d-b552-2874807edb25 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:18.819808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce4630-a489-4a9d-b552-2874807edb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:18.820109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce4630-a489-4a9d-b552-2874807edb25 None admin] PATH_INFO: `/v3/projects/731d452341b64364b192b87761b43850` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:18.821267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce4630-a489-4a9d-b552-2874807edb25 None admin] RBAC: Authorizing `identity:get_project(project_id=731d452341b64364b192b87761b43850)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:18.822690 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce4630-a489-4a9d-b552-2874807edb25 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:18.824781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 221/446] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:18 2022] GET /identity/v3/projects/731d452341b64364b192b87761b43850 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:18.825538 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce4630-a489-4a9d-b552-2874807edb25 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:18.835522 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:18.876040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2fd82e-9659-4a14-a715-3f3d8b067f85 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:18.910047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25248e4-d59b-4319-ab42-e58cf65b94f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:18.910665 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25248e4-d59b-4319-ab42-e58cf65b94f8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:18.910946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25248e4-d59b-4319-ab42-e58cf65b94f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:18.911257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25248e4-d59b-4319-ab42-e58cf65b94f8 None admin] PATH_INFO: `/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:18.912393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25248e4-d59b-4319-ab42-e58cf65b94f8 None admin] RBAC: Authorizing `identity:get_group(group_id=19c6ee7e14484e4e8e8e7af3dbf306d4)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:18.913867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25248e4-d59b-4319-ab42-e58cf65b94f8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:18.915475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 226/447] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:18 2022] GET /identity/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:18.916218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25248e4-d59b-4319-ab42-e58cf65b94f8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:18.925315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce4630-a489-4a9d-b552-2874807edb25 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:18.958547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ad0dc-933a-441c-8ff9-c1ebb1aa7a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:18.959217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ad0dc-933a-441c-8ff9-c1ebb1aa7a97 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:18.959507 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ad0dc-933a-441c-8ff9-c1ebb1aa7a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:18.959785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ad0dc-933a-441c-8ff9-c1ebb1aa7a97 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:18.960501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ad0dc-933a-441c-8ff9-c1ebb1aa7a97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:18.960810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ad0dc-933a-441c-8ff9-c1ebb1aa7a97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:18.962193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ad0dc-933a-441c-8ff9-c1ebb1aa7a97 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:18.968328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:18.971514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 222/448] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:18.972117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ad0dc-933a-441c-8ff9-c1ebb1aa7a97 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:19.023640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25248e4-d59b-4319-ab42-e58cf65b94f8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:19.042410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:19.057754 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea805c0-26e5-49db-b808-5360d0103a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:19.058392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea805c0-26e5-49db-b808-5360d0103a28 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:19.058678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea805c0-26e5-49db-b808-5360d0103a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:19.058957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea805c0-26e5-49db-b808-5360d0103a28 None admin] PATH_INFO: `/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:19.060146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea805c0-26e5-49db-b808-5360d0103a28 None admin] RBAC: Authorizing `identity:get_group(group_id=19c6ee7e14484e4e8e8e7af3dbf306d4)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:19.061608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea805c0-26e5-49db-b808-5360d0103a28 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:19.063245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 227/449] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:18 2022] GET /identity/v3/groups/19c6ee7e14484e4e8e8e7af3dbf306d4 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:19.064000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea805c0-26e5-49db-b808-5360d0103a28 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:19.077419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ad0dc-933a-441c-8ff9-c1ebb1aa7a97 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:19.111139 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e812f-d893-4995-8506-60fc1f7c7bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:19.111757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e812f-d893-4995-8506-60fc1f7c7bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:19.112045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e812f-d893-4995-8506-60fc1f7c7bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:19.112333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e812f-d893-4995-8506-60fc1f7c7bd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:19.113119 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e812f-d893-4995-8506-60fc1f7c7bd2 None admin] RBAC: Adding query filter params (group.id=19c6ee7e14484e4e8e8e7af3dbf306d4, role.id=0a56be0718ea495f8a241ae2447069cf, scope.project.id=731d452341b64364b192b87761b43850) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:19.113434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e812f-d893-4995-8506-60fc1f7c7bd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:19.114877 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e812f-d893-4995-8506-60fc1f7c7bd2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:19.121095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 223/450] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Aug 1 07:24:18 2022] GET /identity/v3/role_assignments?group.id=19c6ee7e14484e4e8e8e7af3dbf306d4&scope.project.id=731d452341b64364b192b87761b43850&role.id=0a56be0718ea495f8a241ae2447069cf => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:19.121796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e812f-d893-4995-8506-60fc1f7c7bd2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:19.173941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea805c0-26e5-49db-b808-5360d0103a28 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:19.188767 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:19.213392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0d9bf-766c-40fb-bad9-91dc4a117352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:19.214653 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d9bf-766c-40fb-bad9-91dc4a117352 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:19.214939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d9bf-766c-40fb-bad9-91dc4a117352 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:19.215251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d9bf-766c-40fb-bad9-91dc4a117352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:19.255738 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e812f-d893-4995-8506-60fc1f7c7bd2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:19.295440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0d9bf-766c-40fb-bad9-91dc4a117352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:19.296077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0d9bf-766c-40fb-bad9-91dc4a117352 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:19.298258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76908c1-f03e-4d3a-b636-701db3659880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:19.298879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76908c1-f03e-4d3a-b636-701db3659880 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:19.299189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76908c1-f03e-4d3a-b636-701db3659880 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:19.299463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76908c1-f03e-4d3a-b636-701db3659880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:19.300247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76908c1-f03e-4d3a-b636-701db3659880 None admin] RBAC: Adding query filter params (group.id=19c6ee7e14484e4e8e8e7af3dbf306d4, role.id=d8e3b1d89cfa4e0fb43b360a31b56605, scope.project.id=731d452341b64364b192b87761b43850) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:19.300555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76908c1-f03e-4d3a-b636-701db3659880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:19.301977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76908c1-f03e-4d3a-b636-701db3659880 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:19.303998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:19.311418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 224/451] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Aug 1 07:24:19 2022] GET /identity/v3/role_assignments?group.id=19c6ee7e14484e4e8e8e7af3dbf306d4&scope.project.id=731d452341b64364b192b87761b43850&role.id=d8e3b1d89cfa4e0fb43b360a31b56605 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:19.409333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 228/452] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:19.415036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76908c1-f03e-4d3a-b636-701db3659880 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:19.517936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76908c1-f03e-4d3a-b636-701db3659880 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:19.551997 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12441c-8a99-4c71-8798-df2d2d061633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:19.552645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12441c-8a99-4c71-8798-df2d2d061633 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:19.552931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12441c-8a99-4c71-8798-df2d2d061633 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:19.553208 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12441c-8a99-4c71-8798-df2d2d061633 None admin] PATH_INFO: `/v3/projects/a9fa482966924d01a3dc7809fa925311` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:19.554368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12441c-8a99-4c71-8798-df2d2d061633 None admin] RBAC: Authorizing `identity:get_project(project_id=a9fa482966924d01a3dc7809fa925311)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:19.555847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12441c-8a99-4c71-8798-df2d2d061633 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:19.557727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 225/453] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:24:19 2022] GET /identity/v3/projects/a9fa482966924d01a3dc7809fa925311 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:19.563237 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0d9bf-766c-40fb-bad9-91dc4a117352 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:19.577667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:19.666315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0d9bf-766c-40fb-bad9-91dc4a117352 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:19.701346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d06cd5-3366-4ff2-9892-9011510eb8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:19.701990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d06cd5-3366-4ff2-9892-9011510eb8bf None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:19.702273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d06cd5-3366-4ff2-9892-9011510eb8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:19.702549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d06cd5-3366-4ff2-9892-9011510eb8bf None admin] PATH_INFO: `/v3/groups/f83c5865c27048f19accb2c86c89d14f` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:19.703744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d06cd5-3366-4ff2-9892-9011510eb8bf None admin] RBAC: Authorizing `identity:get_group(group_id=f83c5865c27048f19accb2c86c89d14f)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:19.705171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d06cd5-3366-4ff2-9892-9011510eb8bf None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:19.706761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 229/454] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Aug 1 07:24:19 2022] GET /identity/v3/groups/f83c5865c27048f19accb2c86c89d14f => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:19.712821 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12441c-8a99-4c71-8798-df2d2d061633 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:19.782208 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:19.816960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12441c-8a99-4c71-8798-df2d2d061633 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:19.849527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574be08-a63f-4030-9660-db8f135609ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:19.850141 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574be08-a63f-4030-9660-db8f135609ae None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:19.850426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574be08-a63f-4030-9660-db8f135609ae None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:19.850703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574be08-a63f-4030-9660-db8f135609ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:19.851517 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574be08-a63f-4030-9660-db8f135609ae None admin] RBAC: Adding query filter params (group.id=f83c5865c27048f19accb2c86c89d14f, role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=a9fa482966924d01a3dc7809fa925311) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:19.851832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574be08-a63f-4030-9660-db8f135609ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:19.853247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574be08-a63f-4030-9660-db8f135609ae None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:19.859536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 226/455] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Aug 1 07:24:19 2022] GET /identity/v3/role_assignments?group.id=f83c5865c27048f19accb2c86c89d14f&scope.project.id=a9fa482966924d01a3dc7809fa925311&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:22.318222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dc97b-8f86-4301-93ed-326dee1ebb7d None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:22.318703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dc97b-8f86-4301-93ed-326dee1ebb7d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:22.318703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dc97b-8f86-4301-93ed-326dee1ebb7d None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:22.320338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 230/456] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:22.326109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c9d08-2b6b-4921-9240-220f54a806ce None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:22.326514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c9d08-2b6b-4921-9240-220f54a806ce None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:22.326673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c9d08-2b6b-4921-9240-220f54a806ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:22.352050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bd9c9d08-2b6b-4921-9240-220f54a806ce None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:22.353246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd9c9d08-2b6b-4921-9240-220f54a806ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:22.369234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a078e-e2b5-480f-9dd1-20011011537b None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:22.369918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a078e-e2b5-480f-9dd1-20011011537b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:22.370396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a078e-e2b5-480f-9dd1-20011011537b None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:22.372401 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 231/457] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:22.377857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb01e29-511a-432b-9d34-c3b003146400 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:22.378390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb01e29-511a-432b-9d34-c3b003146400 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:22.378884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb01e29-511a-432b-9d34-c3b003146400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:22.398539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bbb01e29-511a-432b-9d34-c3b003146400 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:22.400072 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bbb01e29-511a-432b-9d34-c3b003146400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:22.461602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 227/458] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:22.500599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 232/459] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:22.514988 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c9d08-2b6b-4921-9240-220f54a806ce None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:22.525466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e1c14-4ab0-4f6f-866a-fd8eb6aeca44 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:22.528515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e1c14-4ab0-4f6f-866a-fd8eb6aeca44 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:22.529209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e1c14-4ab0-4f6f-866a-fd8eb6aeca44 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:22.532202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 233/460] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:22.533146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e1c14-4ab0-4f6f-866a-fd8eb6aeca44 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:22.546391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:22.596279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:22.653070 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c9d08-2b6b-4921-9240-220f54a806ce None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:22.656061 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e1c14-4ab0-4f6f-866a-fd8eb6aeca44 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:22.695609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89317078-4053-43a2-a19a-d7bc6cfd2998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:22.696677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89317078-4053-43a2-a19a-d7bc6cfd2998 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:22.697153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89317078-4053-43a2-a19a-d7bc6cfd2998 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:22.697718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89317078-4053-43a2-a19a-d7bc6cfd2998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:22.698870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b657fcf-059f-4261-bc0b-6df6b43ed2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:22.699548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b657fcf-059f-4261-bc0b-6df6b43ed2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:22.699838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b657fcf-059f-4261-bc0b-6df6b43ed2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:22.700118 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b657fcf-059f-4261-bc0b-6df6b43ed2e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:22.710226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89317078-4053-43a2-a19a-d7bc6cfd2998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:22.712057 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89317078-4053-43a2-a19a-d7bc6cfd2998 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:22.712431 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b657fcf-059f-4261-bc0b-6df6b43ed2e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:22.715087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b657fcf-059f-4261-bc0b-6df6b43ed2e8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-89317078-4053-43a2-a19a-d7bc6cfd2998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:22.730314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:22.731203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:22.732029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:22.732029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:22.732029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:22.732029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5b657fcf-059f-4261-bc0b-6df6b43ed2e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:22.732241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:22.732960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:22.733799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:22.733799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:22.733799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:22.733799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:22.733799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 228/461] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:22.734891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9075ed-468f-4c2f-9b01-ef6a0c8dae91 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:22.734891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9075ed-468f-4c2f-9b01-ef6a0c8dae91 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:22.735144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9075ed-468f-4c2f-9b01-ef6a0c8dae91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:22.736756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 234/462] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:22.738692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89317078-4053-43a2-a19a-d7bc6cfd2998 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:22.759637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3e9075ed-468f-4c2f-9b01-ef6a0c8dae91 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:22.761790 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e9075ed-468f-4c2f-9b01-ef6a0c8dae91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:22.821651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:22.840491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 229/463] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:22.841779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9075ed-468f-4c2f-9b01-ef6a0c8dae91 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:22.870890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89317078-4053-43a2-a19a-d7bc6cfd2998 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:22.907637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:22.913190 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea178a7-c1e6-469f-b7d0-52d2b11cd882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:22.914191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea178a7-c1e6-469f-b7d0-52d2b11cd882 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:22.914485 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea178a7-c1e6-469f-b7d0-52d2b11cd882 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:22.914769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea178a7-c1e6-469f-b7d0-52d2b11cd882 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:22.915779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea178a7-c1e6-469f-b7d0-52d2b11cd882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:22.916099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea178a7-c1e6-469f-b7d0-52d2b11cd882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:22.918044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea178a7-c1e6-469f-b7d0-52d2b11cd882 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:22.934487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 235/464] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:22.935401 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea178a7-c1e6-469f-b7d0-52d2b11cd882 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:22.984202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9075ed-468f-4c2f-9b01-ef6a0c8dae91 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:23.026458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65469e99-80a6-4ca3-865b-7aaac455fa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:23.027344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65469e99-80a6-4ca3-865b-7aaac455fa4e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:23.027629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65469e99-80a6-4ca3-865b-7aaac455fa4e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:23.027904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65469e99-80a6-4ca3-865b-7aaac455fa4e None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:23.028597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65469e99-80a6-4ca3-865b-7aaac455fa4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:23.028904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65469e99-80a6-4ca3-865b-7aaac455fa4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:23.030293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65469e99-80a6-4ca3-865b-7aaac455fa4e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:23.046637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 230/465] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:23.046637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65469e99-80a6-4ca3-865b-7aaac455fa4e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:23.059507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea178a7-c1e6-469f-b7d0-52d2b11cd882 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:23.083788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:23.097435 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4660d7e-fe13-46db-b4ab-81e4071c4e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:23.098078 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4660d7e-fe13-46db-b4ab-81e4071c4e3f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:23.098362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4660d7e-fe13-46db-b4ab-81e4071c4e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:23.098735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4660d7e-fe13-46db-b4ab-81e4071c4e3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:23.109795 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4660d7e-fe13-46db-b4ab-81e4071c4e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:23.111273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4660d7e-fe13-46db-b4ab-81e4071c4e3f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-f4660d7e-fe13-46db-b4ab-81e4071c4e3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:23.127109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:23.127944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:23.128771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:23.128771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:23.128771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:23.128771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:23.128771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 236/466] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:23.129521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ed46f-daf1-4f03-b632-439e31b26072 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:23.129799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ed46f-daf1-4f03-b632-439e31b26072 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:23.130053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ed46f-daf1-4f03-b632-439e31b26072 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:23.136521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 237/467] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:23.136521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ed46f-daf1-4f03-b632-439e31b26072 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:23.155933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65469e99-80a6-4ca3-865b-7aaac455fa4e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:23.157460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:23.192042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dd4e4-26bd-4a12-83ab-67ba850068ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:23.192654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dd4e4-26bd-4a12-83ab-67ba850068ac None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:23.193054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dd4e4-26bd-4a12-83ab-67ba850068ac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:23.193334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dd4e4-26bd-4a12-83ab-67ba850068ac None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:23.194125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5dd4e4-26bd-4a12-83ab-67ba850068ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:23.195643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5dd4e4-26bd-4a12-83ab-67ba850068ac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:23.227096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 231/468] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:24:23 2022] POST /identity/v3/users => generated 248 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:24:23.227096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dd4e4-26bd-4a12-83ab-67ba850068ac None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:23.259472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ed46f-daf1-4f03-b632-439e31b26072 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:23.309994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967d960-121b-4f73-9eb1-2ba554fccb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:23.311325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967d960-121b-4f73-9eb1-2ba554fccb42 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:23.311933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967d960-121b-4f73-9eb1-2ba554fccb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:23.312227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967d960-121b-4f73-9eb1-2ba554fccb42 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:23.312991 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967d960-121b-4f73-9eb1-2ba554fccb42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:23.315743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967d960-121b-4f73-9eb1-2ba554fccb42 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:23.344079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 238/469] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:24:23 2022] POST /identity/v3/users => generated 246 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:24:23.345445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29ea0e-b186-4ea1-b53f-6def8f20b065 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:23.345740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29ea0e-b186-4ea1-b53f-6def8f20b065 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:23.346020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29ea0e-b186-4ea1-b53f-6def8f20b065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:23.370370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bd29ea0e-b186-4ea1-b53f-6def8f20b065 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:23.371661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bd29ea0e-b186-4ea1-b53f-6def8f20b065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:23.382561 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dd4e4-26bd-4a12-83ab-67ba850068ac None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:23.414868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:23.460583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861b83b-b39a-422e-9310-b3628a23e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:23.461655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861b83b-b39a-422e-9310-b3628a23e33a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:23.462229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861b83b-b39a-422e-9310-b3628a23e33a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:23.462850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861b83b-b39a-422e-9310-b3628a23e33a None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:23.464806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861b83b-b39a-422e-9310-b3628a23e33a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:23.465379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861b83b-b39a-422e-9310-b3628a23e33a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:23.468360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861b83b-b39a-422e-9310-b3628a23e33a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:23.479194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 239/470] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:23.492363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 232/471] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:23.499162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29ea0e-b186-4ea1-b53f-6def8f20b065 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:23.529935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861b83b-b39a-422e-9310-b3628a23e33a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:23.570054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:23.620060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29ea0e-b186-4ea1-b53f-6def8f20b065 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:23.665620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34ef14-3d7f-42b6-bea0-3a6e299464b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:23.667430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34ef14-3d7f-42b6-bea0-3a6e299464b2 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:23.668103 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34ef14-3d7f-42b6-bea0-3a6e299464b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:23.668600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34ef14-3d7f-42b6-bea0-3a6e299464b2 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:23.669748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34ef14-3d7f-42b6-bea0-3a6e299464b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:23.672365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34ef14-3d7f-42b6-bea0-3a6e299464b2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:23.703417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 240/472] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:24:23 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:24:23.744299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861b83b-b39a-422e-9310-b3628a23e33a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:23.813434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0b51-7b11-4df9-9f79-21588188117e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:23.814490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0b51-7b11-4df9-9f79-21588188117e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:23.814954 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0b51-7b11-4df9-9f79-21588188117e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:23.815448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0b51-7b11-4df9-9f79-21588188117e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:23.834242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e0b51-7b11-4df9-9f79-21588188117e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:23.836953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e0b51-7b11-4df9-9f79-21588188117e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ee7e0b51-7b11-4df9-9f79-21588188117e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:23.868637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:23.870256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:23.871271 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:23.871271 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:23.871271 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:23.871271 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:23.873140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 233/473] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:23.882538 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34ef14-3d7f-42b6-bea0-3a6e299464b2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:23.911507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:24.016017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34ef14-3d7f-42b6-bea0-3a6e299464b2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:24.054445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa47e2f-d032-4e20-ae9b-9b14663315f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:24.055077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa47e2f-d032-4e20-ae9b-9b14663315f9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:24.055386 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa47e2f-d032-4e20-ae9b-9b14663315f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:24.055663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa47e2f-d032-4e20-ae9b-9b14663315f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:24.056360 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa47e2f-d032-4e20-ae9b-9b14663315f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:24.056665 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa47e2f-d032-4e20-ae9b-9b14663315f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:24.058098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa47e2f-d032-4e20-ae9b-9b14663315f9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:24.070855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 241/474] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:24.080161 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0b51-7b11-4df9-9f79-21588188117e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:24.159835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:24.195809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0b51-7b11-4df9-9f79-21588188117e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:24.231152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657f408-b462-4514-b54b-c9f0b44de977 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:24.232445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657f408-b462-4514-b54b-c9f0b44de977 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:24.232726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657f408-b462-4514-b54b-c9f0b44de977 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:24.233556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02076a89-a486-4232-a8a8-874c1a80d9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:24.234178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02076a89-a486-4232-a8a8-874c1a80d9ef None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:24.234460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02076a89-a486-4232-a8a8-874c1a80d9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:24.234744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02076a89-a486-4232-a8a8-874c1a80d9ef None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:24.235465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02076a89-a486-4232-a8a8-874c1a80d9ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:24.236656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 242/475] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:24:24.236913 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02076a89-a486-4232-a8a8-874c1a80d9ef None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:24.251420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee81b5-3ff2-4f91-9ba0-0c0e76ba8f7b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:24.252035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee81b5-3ff2-4f91-9ba0-0c0e76ba8f7b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:24.252509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee81b5-3ff2-4f91-9ba0-0c0e76ba8f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:24.275097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 234/476] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:24:24 2022] POST /identity/v3/users => generated 249 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:24:24.280481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-28ee81b5-3ff2-4f91-9ba0-0c0e76ba8f7b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:24.282024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-28ee81b5-3ff2-4f91-9ba0-0c0e76ba8f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:24.368535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 243/477] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:24.403915 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02076a89-a486-4232-a8a8-874c1a80d9ef None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:24.537692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02076a89-a486-4232-a8a8-874c1a80d9ef None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:24.566436 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:24.577875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e577a4-22ca-4c8b-b5f9-be4d11a15a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:24.578554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e577a4-22ca-4c8b-b5f9-be4d11a15a94 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:24.578867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e577a4-22ca-4c8b-b5f9-be4d11a15a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:24.579197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e577a4-22ca-4c8b-b5f9-be4d11a15a94 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:24.579897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e577a4-22ca-4c8b-b5f9-be4d11a15a94 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:24.581989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e577a4-22ca-4c8b-b5f9-be4d11a15a94 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:24.591468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.sql.core [None req-36e577a4-22ca-4c8b-b5f9-be4d11a15a94 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 07:24:24.591468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 07:24:24.591468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [parameters: {'id': '39f8b89286eb43b1b3386da36b71b56f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 07:24:24.591468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80160) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-36e577a4-22ca-4c8b-b5f9-be4d11a15a94 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Aug 01 07:24:24.609093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 07:24:24.610052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application yield resource Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise self.value Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application session.commit() Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 07:24:24.611426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.flush() Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application compat.raise_( Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 07:24:24.612542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 07:24:24.613459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 07:24:24.616249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [parameters: {'id': '39f8b89286eb43b1b3386da36b71b56f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:24.618229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 07:24:24.621518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:24.624933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 235/478] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Aug 1 07:24:24 2022] POST /identity/v3/roles => generated 144 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:24.625545 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee81b5-3ff2-4f91-9ba0-0c0e76ba8f7b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:24.637777 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:24.734418 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee81b5-3ff2-4f91-9ba0-0c0e76ba8f7b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:24.769927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf321263-9cfb-4a45-a10b-b78d1d0f2fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:24.770919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf321263-9cfb-4a45-a10b-b78d1d0f2fdf None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:24.771424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf321263-9cfb-4a45-a10b-b78d1d0f2fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:24.771893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf321263-9cfb-4a45-a10b-b78d1d0f2fdf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:24.782814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf321263-9cfb-4a45-a10b-b78d1d0f2fdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-cf321263-9cfb-4a45-a10b-b78d1d0f2fdf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:24.788325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:24.789202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:24.790011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:24.790011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:24.790011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:24.790011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:24.790011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:24.790011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:24.790011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:24.790011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:24.790011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:24.790011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:24.790011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 07:24:24.790011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:24.792737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 244/479] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:24:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:24.797916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e577a4-22ca-4c8b-b5f9-be4d11a15a94 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:24.913367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e577a4-22ca-4c8b-b5f9-be4d11a15a94 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:24.955128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6ed75-6a17-4938-ab99-b42da439f2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:24.955769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6ed75-6a17-4938-ab99-b42da439f2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:24.956052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6ed75-6a17-4938-ab99-b42da439f2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:24.956330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6ed75-6a17-4938-ab99-b42da439f2d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:24.957065 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6ed75-6a17-4938-ab99-b42da439f2d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:24.957370 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6ed75-6a17-4938-ab99-b42da439f2d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:24.958746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6ed75-6a17-4938-ab99-b42da439f2d5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:24.969562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 236/480] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:25.864710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bbda7-1ebd-42f3-a6d9-51ff2e52bd41 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:25.865256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bbda7-1ebd-42f3-a6d9-51ff2e52bd41 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:25.865256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bbda7-1ebd-42f3-a6d9-51ff2e52bd41 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:25.867011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 245/481] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:25.876378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165bbb0-a086-457c-b398-7d58af98dca8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:25.876817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165bbb0-a086-457c-b398-7d58af98dca8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:25.877016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165bbb0-a086-457c-b398-7d58af98dca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:25.899408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e165bbb0-a086-457c-b398-7d58af98dca8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:25.901157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e165bbb0-a086-457c-b398-7d58af98dca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:25.929303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174028d-5211-45bc-8a67-9fd03e102735 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:25.929934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174028d-5211-45bc-8a67-9fd03e102735 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:25.930317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174028d-5211-45bc-8a67-9fd03e102735 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:25.932113 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 246/482] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:25.938162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc4c5e-cb27-44ee-845a-1c6eaa593d9d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:25.938578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc4c5e-cb27-44ee-845a-1c6eaa593d9d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:25.938964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc4c5e-cb27-44ee-845a-1c6eaa593d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:25.974743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b0cc4c5e-cb27-44ee-845a-1c6eaa593d9d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:25.976005 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b0cc4c5e-cb27-44ee-845a-1c6eaa593d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:25.991474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:26.023540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 237/483] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:26.047460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165bbb0-a086-457c-b398-7d58af98dca8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:26.077688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:26.080760 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 247/484] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:26.164839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc4c5e-cb27-44ee-845a-1c6eaa593d9d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:26.194777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165bbb0-a086-457c-b398-7d58af98dca8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:26.235684 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab542a-af69-4a39-83eb-79a6cf76df58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:26.238006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab542a-af69-4a39-83eb-79a6cf76df58 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:26.238292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab542a-af69-4a39-83eb-79a6cf76df58 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:26.238571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab542a-af69-4a39-83eb-79a6cf76df58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:26.259347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab542a-af69-4a39-83eb-79a6cf76df58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8bab542a-af69-4a39-83eb-79a6cf76df58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:26.267885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:26.268846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:26.269630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:26.269630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:26.269630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:26.269630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:26.269630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:26.269630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:26.269630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:26.269630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:26.269630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:26.269630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:26.269630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:26.269630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:26.270706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 238/485] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:26.278605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab542a-af69-4a39-83eb-79a6cf76df58 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:26.354084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:26.374136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc4c5e-cb27-44ee-845a-1c6eaa593d9d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:26.436992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89559702-55b8-4394-896e-a4323d3565ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:26.438149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89559702-55b8-4394-896e-a4323d3565ac None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:26.438645 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89559702-55b8-4394-896e-a4323d3565ac None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:26.439457 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89559702-55b8-4394-896e-a4323d3565ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:26.445815 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab542a-af69-4a39-83eb-79a6cf76df58 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:26.452826 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89559702-55b8-4394-896e-a4323d3565ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-89559702-55b8-4394-896e-a4323d3565ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:26.459029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:26.460112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:26.460896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:26.460896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:26.460896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:26.460896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:26.460896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:26.460896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:26.460896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:26.460896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:26.460896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:26.460896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:26.460896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:26.460896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:26.468121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 248/486] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:26.468121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622f273-28b9-42df-90b0-8d6f959b8df7 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:26.468121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622f273-28b9-42df-90b0-8d6f959b8df7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:26.468121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622f273-28b9-42df-90b0-8d6f959b8df7 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:26.471139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 249/487] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:26.471864 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622f273-28b9-42df-90b0-8d6f959b8df7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:26.498483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b136a6c-1942-49c9-87ae-646cf1b129fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:26.499190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b136a6c-1942-49c9-87ae-646cf1b129fa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:26.499480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b136a6c-1942-49c9-87ae-646cf1b129fa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:26.499816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b136a6c-1942-49c9-87ae-646cf1b129fa None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:26.500604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b136a6c-1942-49c9-87ae-646cf1b129fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:26.500922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b136a6c-1942-49c9-87ae-646cf1b129fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:26.502319 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b136a6c-1942-49c9-87ae-646cf1b129fa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:26.527339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 239/488] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:26.527339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b43bf8-65f6-40ac-95c9-be808af020bb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:26.527339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b43bf8-65f6-40ac-95c9-be808af020bb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:26.527339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b43bf8-65f6-40ac-95c9-be808af020bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:26.549081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-55b43bf8-65f6-40ac-95c9-be808af020bb None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:26.550589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-55b43bf8-65f6-40ac-95c9-be808af020bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:26.552947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:26.610969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622f273-28b9-42df-90b0-8d6f959b8df7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:26.655506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 240/489] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:26.655506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b43bf8-65f6-40ac-95c9-be808af020bb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:26.670913 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b630a-e08b-41ad-9d24-444a561f5955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:26.672032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b630a-e08b-41ad-9d24-444a561f5955 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:26.672566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b630a-e08b-41ad-9d24-444a561f5955 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:26.673039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b630a-e08b-41ad-9d24-444a561f5955 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:26.674034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b630a-e08b-41ad-9d24-444a561f5955 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:26.674616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b630a-e08b-41ad-9d24-444a561f5955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:26.676328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b630a-e08b-41ad-9d24-444a561f5955 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:26.689276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:26.695354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 250/490] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:26.702245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b630a-e08b-41ad-9d24-444a561f5955 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:26.759112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b43bf8-65f6-40ac-95c9-be808af020bb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:26.798000 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956d0a9-b31f-45f3-b4a6-ef623681315c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:26.798967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956d0a9-b31f-45f3-b4a6-ef623681315c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:26.799395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956d0a9-b31f-45f3-b4a6-ef623681315c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:26.800009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956d0a9-b31f-45f3-b4a6-ef623681315c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:26.814224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956d0a9-b31f-45f3-b4a6-ef623681315c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:26.815611 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b630a-e08b-41ad-9d24-444a561f5955 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:26.815960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956d0a9-b31f-45f3-b4a6-ef623681315c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0956d0a9-b31f-45f3-b4a6-ef623681315c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:26.835770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:26.836825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:26.837838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:26.837838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:26.837838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:26.837838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:26.844633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 241/491] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:26.845022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:26.845802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956d0a9-b31f-45f3-b4a6-ef623681315c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:26.868344 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8333ce-4b05-43e0-96c0-acc85e44740f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:26.869143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8333ce-4b05-43e0-96c0-acc85e44740f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:26.870323 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8333ce-4b05-43e0-96c0-acc85e44740f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:26.870725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8333ce-4b05-43e0-96c0-acc85e44740f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:26.887466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8333ce-4b05-43e0-96c0-acc85e44740f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:26.889064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8333ce-4b05-43e0-96c0-acc85e44740f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-0c8333ce-4b05-43e0-96c0-acc85e44740f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:26.910958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:26.911989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:26.912967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:26.912967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:26.912967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:26.912967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:26.919093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 251/492] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:26.919093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8333ce-4b05-43e0-96c0-acc85e44740f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:26.947382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:26.975269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956d0a9-b31f-45f3-b4a6-ef623681315c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:27.024334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717e7c4-a51b-4f46-98c2-04b530ec4e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:27.025160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717e7c4-a51b-4f46-98c2-04b530ec4e24 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.026115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717e7c4-a51b-4f46-98c2-04b530ec4e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.027565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717e7c4-a51b-4f46-98c2-04b530ec4e24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.040248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8333ce-4b05-43e0-96c0-acc85e44740f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:27.041991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717e7c4-a51b-4f46-98c2-04b530ec4e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c717e7c4-a51b-4f46-98c2-04b530ec4e24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:27.047332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:27.048204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:27.049083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 242/493] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:27.049737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717e7c4-a51b-4f46-98c2-04b530ec4e24 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:27.081584 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be80277-2fef-41a5-addb-0bcf07e5023f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:27.082198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be80277-2fef-41a5-addb-0bcf07e5023f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.082479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be80277-2fef-41a5-addb-0bcf07e5023f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.082756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be80277-2fef-41a5-addb-0bcf07e5023f None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.083492 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be80277-2fef-41a5-addb-0bcf07e5023f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:27.083799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be80277-2fef-41a5-addb-0bcf07e5023f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:27.085177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be80277-2fef-41a5-addb-0bcf07e5023f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:27.097636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 252/494] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:27.102069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f646a-b29d-4598-9818-d09dc1df4011 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.102069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f646a-b29d-4598-9818-d09dc1df4011 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.102069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f646a-b29d-4598-9818-d09dc1df4011 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.102069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 253/495] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:27.102069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f646a-b29d-4598-9818-d09dc1df4011 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:27.135642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:27.172344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717e7c4-a51b-4f46-98c2-04b530ec4e24 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:27.210771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a4683-c9eb-41dd-af8d-218a58e48dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:27.211422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a4683-c9eb-41dd-af8d-218a58e48dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.211691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f646a-b29d-4598-9818-d09dc1df4011 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:27.211793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a4683-c9eb-41dd-af8d-218a58e48dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.211980 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a4683-c9eb-41dd-af8d-218a58e48dd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.212670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a4683-c9eb-41dd-af8d-218a58e48dd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:27.213502 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a4683-c9eb-41dd-af8d-218a58e48dd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:27.215725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a4683-c9eb-41dd-af8d-218a58e48dd0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:27.228052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 243/496] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:27.233536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a4683-c9eb-41dd-af8d-218a58e48dd0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:27.241772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:27.251640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ac889-5ce0-4b1d-b86c-7ddf6b8f4258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:27.252264 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac889-5ce0-4b1d-b86c-7ddf6b8f4258 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.252547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac889-5ce0-4b1d-b86c-7ddf6b8f4258 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.253045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac889-5ce0-4b1d-b86c-7ddf6b8f4258 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.253529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ac889-5ce0-4b1d-b86c-7ddf6b8f4258 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:27.253834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ac889-5ce0-4b1d-b86c-7ddf6b8f4258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:27.255222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ac889-5ce0-4b1d-b86c-7ddf6b8f4258 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:27.264868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 254/497] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:27.265921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c2f6c-2caa-4505-9a75-f383ff308fac None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.266209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c2f6c-2caa-4505-9a75-f383ff308fac None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.266487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c2f6c-2caa-4505-9a75-f383ff308fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.285818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-882c2f6c-2caa-4505-9a75-f383ff308fac None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:27.287027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-882c2f6c-2caa-4505-9a75-f383ff308fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:27.342975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a4683-c9eb-41dd-af8d-218a58e48dd0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:27.363131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 255/498] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:27.363131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c2f6c-2caa-4505-9a75-f383ff308fac None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:27.391883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:27.395968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789bff0-7c13-48f6-801b-6def2a256567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:27.396688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789bff0-7c13-48f6-801b-6def2a256567 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.396900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789bff0-7c13-48f6-801b-6def2a256567 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.397220 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789bff0-7c13-48f6-801b-6def2a256567 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.405332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789bff0-7c13-48f6-801b-6def2a256567 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:27.406799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789bff0-7c13-48f6-801b-6def2a256567 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e789bff0-7c13-48f6-801b-6def2a256567 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:27.420545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:27.421451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:27.422327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:27.422327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:27.422327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:27.422327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:27.422327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:27.422327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:27.422327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:27.422327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:27.422327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:24:27.422327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:27.422327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 244/499] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Aug 1 07:24:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:27.422873 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789bff0-7c13-48f6-801b-6def2a256567 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:27.484543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c2f6c-2caa-4505-9a75-f383ff308fac None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:27.495770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:27.522067 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30f03c-18a5-480c-8eeb-2f0720f085e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:27.522716 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30f03c-18a5-480c-8eeb-2f0720f085e0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.523000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30f03c-18a5-480c-8eeb-2f0720f085e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.523313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30f03c-18a5-480c-8eeb-2f0720f085e0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.530993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30f03c-18a5-480c-8eeb-2f0720f085e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:27.532150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789bff0-7c13-48f6-801b-6def2a256567 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:27.532461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30f03c-18a5-480c-8eeb-2f0720f085e0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-de30f03c-18a5-480c-8eeb-2f0720f085e0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:27.546266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:27.547238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:27.548136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:27.548136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:27.548136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:27.548136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:27.548136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:27.548136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:27.548136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:27.548136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:27.548136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 07:24:27.548136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:27.548136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 256/500] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:27.548700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30f03c-18a5-480c-8eeb-2f0720f085e0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:27.574333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0238a0b-bf89-415b-8520-0db755af476e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:27.574978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0238a0b-bf89-415b-8520-0db755af476e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.575294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0238a0b-bf89-415b-8520-0db755af476e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.575581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0238a0b-bf89-415b-8520-0db755af476e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.586161 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0238a0b-bf89-415b-8520-0db755af476e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:27.587675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0238a0b-bf89-415b-8520-0db755af476e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b0238a0b-bf89-415b-8520-0db755af476e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:27.601900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:27.602969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:27.604735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:27.604735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:27.604735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:27.604735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:27.604735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 245/501] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:27.604735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0238a0b-bf89-415b-8520-0db755af476e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:27.649191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:27.668733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30f03c-18a5-480c-8eeb-2f0720f085e0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:27.702832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236d39b-ef72-428b-95c4-2c7ff8af4b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:27.703492 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236d39b-ef72-428b-95c4-2c7ff8af4b92 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.703774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236d39b-ef72-428b-95c4-2c7ff8af4b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.704049 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236d39b-ef72-428b-95c4-2c7ff8af4b92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.706041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0238a0b-bf89-415b-8520-0db755af476e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:27.714443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236d39b-ef72-428b-95c4-2c7ff8af4b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-b236d39b-ef72-428b-95c4-2c7ff8af4b92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:27.719895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:27.720851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:27.721694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 257/502] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:27.722348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236d39b-ef72-428b-95c4-2c7ff8af4b92 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:27.726999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:27.745595 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e87d8-90d6-4e37-95b3-ae97dc9abac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:27.746139 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e87d8-90d6-4e37-95b3-ae97dc9abac0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.746421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e87d8-90d6-4e37-95b3-ae97dc9abac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.746702 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e87d8-90d6-4e37-95b3-ae97dc9abac0 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.747531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e87d8-90d6-4e37-95b3-ae97dc9abac0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:27.747838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e87d8-90d6-4e37-95b3-ae97dc9abac0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:27.749268 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e87d8-90d6-4e37-95b3-ae97dc9abac0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:27.785157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 246/503] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Aug 1 07:24:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:27.786043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e87d8-90d6-4e37-95b3-ae97dc9abac0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:27.822740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236d39b-ef72-428b-95c4-2c7ff8af4b92 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:27.856527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00205524-276d-4aba-ab42-34637cc66d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:27.857135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00205524-276d-4aba-ab42-34637cc66d14 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.857417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00205524-276d-4aba-ab42-34637cc66d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.857693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00205524-276d-4aba-ab42-34637cc66d14 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.858483 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00205524-276d-4aba-ab42-34637cc66d14 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:27.858792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00205524-276d-4aba-ab42-34637cc66d14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:27.860257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00205524-276d-4aba-ab42-34637cc66d14 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:27.882797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e87d8-90d6-4e37-95b3-ae97dc9abac0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:27.892175 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:27.896907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 258/504] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Aug 1 07:24:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:27.898105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffb2ed-9871-4b02-907b-2a05293e579c None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.898392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffb2ed-9871-4b02-907b-2a05293e579c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.898671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffb2ed-9871-4b02-907b-2a05293e579c None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.899908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 259/505] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:27.903257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffb2ed-9871-4b02-907b-2a05293e579c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:27.920225 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83ed9b-cabf-4ed7-bef7-307ca829ca98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:27.920853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83ed9b-cabf-4ed7-bef7-307ca829ca98 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:27.921140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83ed9b-cabf-4ed7-bef7-307ca829ca98 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:27.921422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83ed9b-cabf-4ed7-bef7-307ca829ca98 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:27.922113 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83ed9b-cabf-4ed7-bef7-307ca829ca98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:27.922422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83ed9b-cabf-4ed7-bef7-307ca829ca98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:27.923850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83ed9b-cabf-4ed7-bef7-307ca829ca98 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:27.935556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 247/506] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:27.936350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83ed9b-cabf-4ed7-bef7-307ca829ca98 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:27.976243 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:28.011031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffb2ed-9871-4b02-907b-2a05293e579c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:28.044557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6be92-2030-4911-b9a6-529db132563b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:28.045242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6be92-2030-4911-b9a6-529db132563b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:28.045532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6be92-2030-4911-b9a6-529db132563b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:28.045811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6be92-2030-4911-b9a6-529db132563b None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:28.046397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83ed9b-cabf-4ed7-bef7-307ca829ca98 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:28.046557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6be92-2030-4911-b9a6-529db132563b None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:28.046834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6be92-2030-4911-b9a6-529db132563b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:28.048258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6be92-2030-4911-b9a6-529db132563b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:28.056156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 260/507] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:28.057494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6be92-2030-4911-b9a6-529db132563b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:28.081077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79faf8-a699-4e3f-a2cf-ba6b85f3858d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:28.081695 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79faf8-a699-4e3f-a2cf-ba6b85f3858d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:28.081981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79faf8-a699-4e3f-a2cf-ba6b85f3858d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:28.082260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79faf8-a699-4e3f-a2cf-ba6b85f3858d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:28.118457 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:28.151315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79faf8-a699-4e3f-a2cf-ba6b85f3858d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:28.152779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79faf8-a699-4e3f-a2cf-ba6b85f3858d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:28.158055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6be92-2030-4911-b9a6-529db132563b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:28.186500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:28.192324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53adfe9-368c-4d34-8e49-190689721086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:28.192935 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53adfe9-368c-4d34-8e49-190689721086 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:28.193224 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53adfe9-368c-4d34-8e49-190689721086 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:28.193501 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53adfe9-368c-4d34-8e49-190689721086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:28.251937 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 248/508] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:28.253175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d0a3a-7d49-44f0-8ca8-e0d9364d4edf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:28.253455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d0a3a-7d49-44f0-8ca8-e0d9364d4edf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:28.253739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d0a3a-7d49-44f0-8ca8-e0d9364d4edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:28.262047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53adfe9-368c-4d34-8e49-190689721086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:28.263499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53adfe9-368c-4d34-8e49-190689721086 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:28.270795 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a78d0a3a-7d49-44f0-8ca8-e0d9364d4edf None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:28.272051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a78d0a3a-7d49-44f0-8ca8-e0d9364d4edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:28.322684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:28.336688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 249/509] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:28.337513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d0a3a-7d49-44f0-8ca8-e0d9364d4edf None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:28.366081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 261/510] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:28.366081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53adfe9-368c-4d34-8e49-190689721086 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:28.407128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:28.444536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d0a3a-7d49-44f0-8ca8-e0d9364d4edf None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:28.464373 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53adfe9-368c-4d34-8e49-190689721086 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:28.477633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece3a20-1bad-41e4-84ae-04b429048dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:28.478278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece3a20-1bad-41e4-84ae-04b429048dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:28.479139 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece3a20-1bad-41e4-84ae-04b429048dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:28.479638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece3a20-1bad-41e4-84ae-04b429048dd3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:28.486556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece3a20-1bad-41e4-84ae-04b429048dd3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:28.488034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece3a20-1bad-41e4-84ae-04b429048dd3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:28.498981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd822e6c-5619-4a01-9049-dfcad99f1177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:28.499638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd822e6c-5619-4a01-9049-dfcad99f1177 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:28.499926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd822e6c-5619-4a01-9049-dfcad99f1177 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:28.500204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd822e6c-5619-4a01-9049-dfcad99f1177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0ece3a20-1bad-41e4-84ae-04b429048dd3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:28.500338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:28.501248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:28.502150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:28.502150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:28.502150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:28.502150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:28.502150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:28.502150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:28.502150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:28.502150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:28.502150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:24:28.502150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:28.502150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 250/511] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:28.502731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece3a20-1bad-41e4-84ae-04b429048dd3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:28.512537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd822e6c-5619-4a01-9049-dfcad99f1177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:28.513910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd822e6c-5619-4a01-9049-dfcad99f1177 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-cd822e6c-5619-4a01-9049-dfcad99f1177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:28.529368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:28.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:28.531259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:28.531259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:28.531259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:28.531259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:28.531259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 262/512] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:28.531513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd822e6c-5619-4a01-9049-dfcad99f1177 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:28.548202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:28.613987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece3a20-1bad-41e4-84ae-04b429048dd3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:28.630852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd822e6c-5619-4a01-9049-dfcad99f1177 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:28.639476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:28.648330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b4313-a9c0-4dbc-9c10-091c55ff0385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:28.648957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b4313-a9c0-4dbc-9c10-091c55ff0385 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:28.649240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b4313-a9c0-4dbc-9c10-091c55ff0385 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:28.649516 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b4313-a9c0-4dbc-9c10-091c55ff0385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:28.660461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b4313-a9c0-4dbc-9c10-091c55ff0385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:28.661899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b4313-a9c0-4dbc-9c10-091c55ff0385 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:28.665493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23880f2a-8ce6-4c61-9793-f3432335335f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:28.666708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23880f2a-8ce6-4c61-9793-f3432335335f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:28.666996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23880f2a-8ce6-4c61-9793-f3432335335f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:28.667306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23880f2a-8ce6-4c61-9793-f3432335335f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fb1b4313-a9c0-4dbc-9c10-091c55ff0385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:28.676197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:28.677032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:28.677883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:28.677883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:28.677883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:28.677883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:28.677883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 251/513] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:28.678147 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23880f2a-8ce6-4c61-9793-f3432335335f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:28.678213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b4313-a9c0-4dbc-9c10-091c55ff0385 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:28.679150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23880f2a-8ce6-4c61-9793-f3432335335f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-23880f2a-8ce6-4c61-9793-f3432335335f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:28.696259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:28.697112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:28.698108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:28.698108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:28.698108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:28.698108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:28.698108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 263/514] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:28.698400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23880f2a-8ce6-4c61-9793-f3432335335f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:28.761976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:28.782267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b4313-a9c0-4dbc-9c10-091c55ff0385 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:28.797118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23880f2a-8ce6-4c61-9793-f3432335335f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:28.815575 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2919a-c1d5-4f46-8519-a9fa9b41f15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:28.816211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2919a-c1d5-4f46-8519-a9fa9b41f15c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:28.816504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2919a-c1d5-4f46-8519-a9fa9b41f15c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:28.816785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2919a-c1d5-4f46-8519-a9fa9b41f15c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:28.827994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2919a-c1d5-4f46-8519-a9fa9b41f15c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:28.829464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2919a-c1d5-4f46-8519-a9fa9b41f15c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:28.831254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294963c-a4f4-4b76-83b4-e63cee2f3a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:28.831862 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294963c-a4f4-4b76-83b4-e63cee2f3a43 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:28.832145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294963c-a4f4-4b76-83b4-e63cee2f3a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:28.832421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294963c-a4f4-4b76-83b4-e63cee2f3a43 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:28.833229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294963c-a4f4-4b76-83b4-e63cee2f3a43 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:28.833539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294963c-a4f4-4b76-83b4-e63cee2f3a43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:28.834954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294963c-a4f4-4b76-83b4-e63cee2f3a43 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-30d2919a-c1d5-4f46-8519-a9fa9b41f15c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:28.843468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:28.844360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:28.845186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:28.845186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:28.845186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:28.845186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:28.845186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 252/515] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:28.845452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2919a-c1d5-4f46-8519-a9fa9b41f15c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:28.856630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 264/516] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Aug 1 07:24:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:28.857439 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294963c-a4f4-4b76-83b4-e63cee2f3a43 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:28.876346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:28.945218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2919a-c1d5-4f46-8519-a9fa9b41f15c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:28.957596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294963c-a4f4-4b76-83b4-e63cee2f3a43 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:28.979450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652ac0a-7550-49d6-872f-8804587d96d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:28.980058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652ac0a-7550-49d6-872f-8804587d96d6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:28.980351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652ac0a-7550-49d6-872f-8804587d96d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:28.980630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652ac0a-7550-49d6-872f-8804587d96d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:28.981313 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652ac0a-7550-49d6-872f-8804587d96d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:28.981620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652ac0a-7550-49d6-872f-8804587d96d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:28.982991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652ac0a-7550-49d6-872f-8804587d96d6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:28.983756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:28.995374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252ae4e-ee10-4763-a7b4-b7c1d7301cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:28.995795 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 253/517] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:28.995996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252ae4e-ee10-4763-a7b4-b7c1d7301cdc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:28.996272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252ae4e-ee10-4763-a7b4-b7c1d7301cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:28.996547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252ae4e-ee10-4763-a7b4-b7c1d7301cdc None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:28.996641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652ac0a-7550-49d6-872f-8804587d96d6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:28.997235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252ae4e-ee10-4763-a7b4-b7c1d7301cdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:28.997542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252ae4e-ee10-4763-a7b4-b7c1d7301cdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:28.998930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252ae4e-ee10-4763-a7b4-b7c1d7301cdc None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:29.010972 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 265/518] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:29.011591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252ae4e-ee10-4763-a7b4-b7c1d7301cdc None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:29.090430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:29.101013 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652ac0a-7550-49d6-872f-8804587d96d6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:29.107808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252ae4e-ee10-4763-a7b4-b7c1d7301cdc None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:29.134213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2ae2d-455b-4405-9f1e-1ec7ea4989fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:29.134858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2ae2d-455b-4405-9f1e-1ec7ea4989fc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:29.135175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2ae2d-455b-4405-9f1e-1ec7ea4989fc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:29.135766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2ae2d-455b-4405-9f1e-1ec7ea4989fc None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:29.136352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2ae2d-455b-4405-9f1e-1ec7ea4989fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:29.136657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2ae2d-455b-4405-9f1e-1ec7ea4989fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:29.138075 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2ae2d-455b-4405-9f1e-1ec7ea4989fc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:29.141125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c272f-a4f4-4625-acf0-44ae84e9f495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:29.141729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c272f-a4f4-4625-acf0-44ae84e9f495 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:29.142015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c272f-a4f4-4625-acf0-44ae84e9f495 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:29.142290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c272f-a4f4-4625-acf0-44ae84e9f495 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:29.142972 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c272f-a4f4-4625-acf0-44ae84e9f495 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:29.143312 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c272f-a4f4-4625-acf0-44ae84e9f495 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:29.144681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c272f-a4f4-4625-acf0-44ae84e9f495 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:29.148889 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 254/519] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:29.149726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2ae2d-455b-4405-9f1e-1ec7ea4989fc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:29.159294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 266/520] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:29.160853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c272f-a4f4-4625-acf0-44ae84e9f495 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:29.214213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:29.253465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2ae2d-455b-4405-9f1e-1ec7ea4989fc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:29.278193 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c272f-a4f4-4625-acf0-44ae84e9f495 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:29.286116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bbea3-a06e-4079-8f55-aa919642fe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:29.286727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bbea3-a06e-4079-8f55-aa919642fe5e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:29.287008 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bbea3-a06e-4079-8f55-aa919642fe5e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:29.287309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bbea3-a06e-4079-8f55-aa919642fe5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:29.305700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:29.318544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef86008-3e86-4451-bcd1-8f70b98c88ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:29.319403 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef86008-3e86-4451-bcd1-8f70b98c88ba None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:29.319847 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef86008-3e86-4451-bcd1-8f70b98c88ba None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:29.320287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef86008-3e86-4451-bcd1-8f70b98c88ba None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:29.329840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef86008-3e86-4451-bcd1-8f70b98c88ba None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:29.331827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef86008-3e86-4451-bcd1-8f70b98c88ba None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-0ef86008-3e86-4451-bcd1-8f70b98c88ba None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:29.349313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:29.350212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:29.351143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:29.351143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:29.351143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:29.351143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:29.351143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:29.351143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:29.351143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:29.351143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:29.351143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 07:24:29.351143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:29.351143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 267/521] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:29.352301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef86008-3e86-4451-bcd1-8f70b98c88ba None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:29.358470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bbea3-a06e-4079-8f55-aa919642fe5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:29.360017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bbea3-a06e-4079-8f55-aa919642fe5e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:29.458503 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 255/522] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:29.459356 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bbea3-a06e-4079-8f55-aa919642fe5e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:29.475522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef86008-3e86-4451-bcd1-8f70b98c88ba None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:29.479085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:29.507744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:29.517667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169a30b-0c25-461c-9e7d-ee7330b5d64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:29.518491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169a30b-0c25-461c-9e7d-ee7330b5d64c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:29.518932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169a30b-0c25-461c-9e7d-ee7330b5d64c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:29.519398 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169a30b-0c25-461c-9e7d-ee7330b5d64c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:29.528680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169a30b-0c25-461c-9e7d-ee7330b5d64c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:29.530683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169a30b-0c25-461c-9e7d-ee7330b5d64c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-c169a30b-0c25-461c-9e7d-ee7330b5d64c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:29.548203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:29.549104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:29.549930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:29.549930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:29.549930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:29.549930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:29.549930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:29.549930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:29.549930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:29.549930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:29.549930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 268/523] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:29.550562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169a30b-0c25-461c-9e7d-ee7330b5d64c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:29.562115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bbea3-a06e-4079-8f55-aa919642fe5e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:29.595331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96059527-6ad6-4077-9ebe-7577fa098521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:29.595937 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96059527-6ad6-4077-9ebe-7577fa098521 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:29.596221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96059527-6ad6-4077-9ebe-7577fa098521 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:29.596501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96059527-6ad6-4077-9ebe-7577fa098521 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:29.597170 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96059527-6ad6-4077-9ebe-7577fa098521 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:29.598615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96059527-6ad6-4077-9ebe-7577fa098521 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:29.620255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 256/524] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:24:29 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:24:29.621104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96059527-6ad6-4077-9ebe-7577fa098521 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:29.657642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169a30b-0c25-461c-9e7d-ee7330b5d64c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:29.694308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8274b4-906d-4cb7-bd90-6f0601ecd2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:29.694988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8274b4-906d-4cb7-bd90-6f0601ecd2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:29.695302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8274b4-906d-4cb7-bd90-6f0601ecd2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:29.695584 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8274b4-906d-4cb7-bd90-6f0601ecd2f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:29.703110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8274b4-906d-4cb7-bd90-6f0601ecd2f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:29.704627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8274b4-906d-4cb7-bd90-6f0601ecd2f9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-cf8274b4-906d-4cb7-bd90-6f0601ecd2f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:29.717085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:29.718179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:29.719255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:29.719255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:29.719255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:29.719255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:29.719255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:29.719255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:29.719255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:29.719255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:29.719255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 269/525] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:29.719869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8274b4-906d-4cb7-bd90-6f0601ecd2f9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:29.749451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96059527-6ad6-4077-9ebe-7577fa098521 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:29.754736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:29.757198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:29.793944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276d2fd-a2e2-41b5-a286-3d6eb401695d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:29.794571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276d2fd-a2e2-41b5-a286-3d6eb401695d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:29.794859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276d2fd-a2e2-41b5-a286-3d6eb401695d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:29.795170 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276d2fd-a2e2-41b5-a286-3d6eb401695d None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:29.795975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276d2fd-a2e2-41b5-a286-3d6eb401695d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:29.796287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276d2fd-a2e2-41b5-a286-3d6eb401695d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:29.797727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276d2fd-a2e2-41b5-a286-3d6eb401695d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:29.820034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 257/526] 10.222.0.40 () {56 vars in 1139 bytes} [Mon Aug 1 07:24:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:29.820886 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276d2fd-a2e2-41b5-a286-3d6eb401695d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:29.843315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8274b4-906d-4cb7-bd90-6f0601ecd2f9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:29.881186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8b021-1261-43e9-badb-59bc9c818b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:29.881773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8b021-1261-43e9-badb-59bc9c818b34 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:29.882133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8b021-1261-43e9-badb-59bc9c818b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:29.882416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8b021-1261-43e9-badb-59bc9c818b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:29.895749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8b021-1261-43e9-badb-59bc9c818b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:29.897230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8b021-1261-43e9-badb-59bc9c818b34 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-3db8b021-1261-43e9-badb-59bc9c818b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:29.913191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:29.914093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:29.914925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:29.914925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:29.914925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:29.914925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:29.914925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 270/527] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:29.915836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8b021-1261-43e9-badb-59bc9c818b34 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:29.926605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276d2fd-a2e2-41b5-a286-3d6eb401695d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:29.967228 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12007cd-0f3c-42b1-add8-a125fd592343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:29.967861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12007cd-0f3c-42b1-add8-a125fd592343 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:29.968150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12007cd-0f3c-42b1-add8-a125fd592343 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:29.968425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12007cd-0f3c-42b1-add8-a125fd592343 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:29.969433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12007cd-0f3c-42b1-add8-a125fd592343 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:29.969562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12007cd-0f3c-42b1-add8-a125fd592343 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:29.970892 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12007cd-0f3c-42b1-add8-a125fd592343 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:29.995241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 258/528] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:29.996148 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12007cd-0f3c-42b1-add8-a125fd592343 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:30.007998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:30.023723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:30.033894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8b021-1261-43e9-badb-59bc9c818b34 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:30.070709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8f520-4566-4ec3-bf14-9ea0665aebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:30.071357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8f520-4566-4ec3-bf14-9ea0665aebef None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:30.071648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8f520-4566-4ec3-bf14-9ea0665aebef None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:30.072084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8f520-4566-4ec3-bf14-9ea0665aebef None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:30.073483 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8f520-4566-4ec3-bf14-9ea0665aebef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:30.073802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8f520-4566-4ec3-bf14-9ea0665aebef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:30.075271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8f520-4566-4ec3-bf14-9ea0665aebef None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:30.093800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 271/529] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:30.094432 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8f520-4566-4ec3-bf14-9ea0665aebef None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:30.112003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12007cd-0f3c-42b1-add8-a125fd592343 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:30.147386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608d801-c43e-4a45-a64a-c10aeb3aecf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:30.147916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608d801-c43e-4a45-a64a-c10aeb3aecf2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:30.148269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608d801-c43e-4a45-a64a-c10aeb3aecf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:30.148544 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608d801-c43e-4a45-a64a-c10aeb3aecf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:30.210797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8f520-4566-4ec3-bf14-9ea0665aebef None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:30.223404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608d801-c43e-4a45-a64a-c10aeb3aecf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:30.224868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608d801-c43e-4a45-a64a-c10aeb3aecf2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:30.247592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18be43e-2f68-4701-91ff-28c002fab305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:30.248304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18be43e-2f68-4701-91ff-28c002fab305 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:30.248590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18be43e-2f68-4701-91ff-28c002fab305 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:30.248870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18be43e-2f68-4701-91ff-28c002fab305 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:30.249560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18be43e-2f68-4701-91ff-28c002fab305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:30.249867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18be43e-2f68-4701-91ff-28c002fab305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:30.251255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18be43e-2f68-4701-91ff-28c002fab305 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:30.252913 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:30.254475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:30.265620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 272/530] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:30.266155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18be43e-2f68-4701-91ff-28c002fab305 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:30.341343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 259/531] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:30.341343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608d801-c43e-4a45-a64a-c10aeb3aecf2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:30.374287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18be43e-2f68-4701-91ff-28c002fab305 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:30.409593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc6bca-bbb3-48ff-8e3a-4b125b9f47db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:30.410236 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc6bca-bbb3-48ff-8e3a-4b125b9f47db None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:30.410519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc6bca-bbb3-48ff-8e3a-4b125b9f47db None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:30.410799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc6bca-bbb3-48ff-8e3a-4b125b9f47db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:30.411614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc6bca-bbb3-48ff-8e3a-4b125b9f47db None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=abdcd6d09fee45c696449641ea36eae3) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:30.411922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc6bca-bbb3-48ff-8e3a-4b125b9f47db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:30.413367 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc6bca-bbb3-48ff-8e3a-4b125b9f47db None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:30.418924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 273/532] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:30 2022] GET /identity/v3/role_assignments?user.id=abdcd6d09fee45c696449641ea36eae3&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:30.424086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc6bca-bbb3-48ff-8e3a-4b125b9f47db None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:30.461346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608d801-c43e-4a45-a64a-c10aeb3aecf2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:30.497364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:30.498015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:30.516050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b62473-964b-456d-9112-ce2ece2f8fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:30.516708 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b62473-964b-456d-9112-ce2ece2f8fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:30.517006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b62473-964b-456d-9112-ce2ece2f8fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:30.517291 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b62473-964b-456d-9112-ce2ece2f8fe4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:30.518106 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b62473-964b-456d-9112-ce2ece2f8fe4 None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=5f57fa108c3d478a9976f4aea5f33bb5) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:30.518426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b62473-964b-456d-9112-ce2ece2f8fe4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:30.521072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b62473-964b-456d-9112-ce2ece2f8fe4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:30.539533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 260/533] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:30 2022] GET /identity/v3/role_assignments?user.id=5f57fa108c3d478a9976f4aea5f33bb5&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:30.540903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b62473-964b-456d-9112-ce2ece2f8fe4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:30.563088 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc6bca-bbb3-48ff-8e3a-4b125b9f47db None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:30.605085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e5bf6-1498-4263-a836-8c4e571c4a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:30.605745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e5bf6-1498-4263-a836-8c4e571c4a9e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:30.606031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e5bf6-1498-4263-a836-8c4e571c4a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:30.606357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e5bf6-1498-4263-a836-8c4e571c4a9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:30.627725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e5bf6-1498-4263-a836-8c4e571c4a9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:30.632443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e5bf6-1498-4263-a836-8c4e571c4a9e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-889e5bf6-1498-4263-a836-8c4e571c4a9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:30.660458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:30.661327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:30.662148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:30.662148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:30.662148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:30.662148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:30.662148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:30.662148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:30.662148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:30.662148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:30.664259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 274/534] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:30.683032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e5bf6-1498-4263-a836-8c4e571c4a9e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:30.722156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b62473-964b-456d-9112-ce2ece2f8fe4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:30.766445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c13f8b-2e4f-4ba5-8e3a-70f4baffe518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:30.766445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c13f8b-2e4f-4ba5-8e3a-70f4baffe518 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:30.766445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c13f8b-2e4f-4ba5-8e3a-70f4baffe518 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:30.766445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c13f8b-2e4f-4ba5-8e3a-70f4baffe518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:30.781733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c13f8b-2e4f-4ba5-8e3a-70f4baffe518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:30.783844 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c13f8b-2e4f-4ba5-8e3a-70f4baffe518 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-50c13f8b-2e4f-4ba5-8e3a-70f4baffe518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:30.808098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:30.809167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:30.810183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:30.810183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:30.810183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:30.810183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:30.810183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 261/535] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:30.813095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c13f8b-2e4f-4ba5-8e3a-70f4baffe518 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:30.867234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:30.870569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e5bf6-1498-4263-a836-8c4e571c4a9e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:30.894223 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:30.927191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2e7cb-cc44-4e8b-9a81-0f8972955ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:30.928152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2e7cb-cc44-4e8b-9a81-0f8972955ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:30.928610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2e7cb-cc44-4e8b-9a81-0f8972955ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:30.929079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2e7cb-cc44-4e8b-9a81-0f8972955ff4 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:30.930106 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2e7cb-cc44-4e8b-9a81-0f8972955ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:30.930426 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2e7cb-cc44-4e8b-9a81-0f8972955ff4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:30.932553 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2e7cb-cc44-4e8b-9a81-0f8972955ff4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:30.958990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c13f8b-2e4f-4ba5-8e3a-70f4baffe518 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:30.959678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 275/536] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:30.964055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2e7cb-cc44-4e8b-9a81-0f8972955ff4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:31.022423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2b71a-d666-42cc-82aa-e52ca0793793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:31.024533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2b71a-d666-42cc-82aa-e52ca0793793 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:31.026278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2b71a-d666-42cc-82aa-e52ca0793793 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:31.026764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2b71a-d666-42cc-82aa-e52ca0793793 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:31.028983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2b71a-d666-42cc-82aa-e52ca0793793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:31.029508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2b71a-d666-42cc-82aa-e52ca0793793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:31.032043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2b71a-d666-42cc-82aa-e52ca0793793 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:31.057946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 262/537] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:31.063932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2b71a-d666-42cc-82aa-e52ca0793793 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:31.176880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:31.183386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2b71a-d666-42cc-82aa-e52ca0793793 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:31.194198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2e7cb-cc44-4e8b-9a81-0f8972955ff4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:31.249417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19399467-6b68-48af-9b27-66cde1417a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:31.250071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19399467-6b68-48af-9b27-66cde1417a62 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:31.250355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19399467-6b68-48af-9b27-66cde1417a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:31.250633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19399467-6b68-48af-9b27-66cde1417a62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:31.262600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47042a0a-5d1c-458b-9df5-973bfb099843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:31.264351 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47042a0a-5d1c-458b-9df5-973bfb099843 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:31.264642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47042a0a-5d1c-458b-9df5-973bfb099843 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:31.264923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47042a0a-5d1c-458b-9df5-973bfb099843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:31.265729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47042a0a-5d1c-458b-9df5-973bfb099843 None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=495c489f00d74b53ac02c2cbbb323c38) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:31.266045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47042a0a-5d1c-458b-9df5-973bfb099843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:31.267517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47042a0a-5d1c-458b-9df5-973bfb099843 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:31.271614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19399467-6b68-48af-9b27-66cde1417a62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:31.273421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19399467-6b68-48af-9b27-66cde1417a62 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:31.287224 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 276/538] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:30 2022] GET /identity/v3/role_assignments?user.id=495c489f00d74b53ac02c2cbbb323c38&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-19399467-6b68-48af-9b27-66cde1417a62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:31.296736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:31.297576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:31.298408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:31.298408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:31.298408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:31.298408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:31.298408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:31.298408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:31.298408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:31.298408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:31.302571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 263/539] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:31.303947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47042a0a-5d1c-458b-9df5-973bfb099843 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:31.374906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:31.451758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47042a0a-5d1c-458b-9df5-973bfb099843 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:31.504627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644662e-b84b-4ac7-9531-9593072bc451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:31.505401 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644662e-b84b-4ac7-9531-9593072bc451 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:31.505692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644662e-b84b-4ac7-9531-9593072bc451 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:31.505975 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644662e-b84b-4ac7-9531-9593072bc451 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:31.506811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644662e-b84b-4ac7-9531-9593072bc451 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:31.507155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644662e-b84b-4ac7-9531-9593072bc451 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:31.508913 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644662e-b84b-4ac7-9531-9593072bc451 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:31.527250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 277/540] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:31.529667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19399467-6b68-48af-9b27-66cde1417a62 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:31.651311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19399467-6b68-48af-9b27-66cde1417a62 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:31.688903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dface367-dafe-4c34-bd14-3f733677bb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:31.689611 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dface367-dafe-4c34-bd14-3f733677bb80 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:31.689905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dface367-dafe-4c34-bd14-3f733677bb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:31.690188 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dface367-dafe-4c34-bd14-3f733677bb80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:31.691004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dface367-dafe-4c34-bd14-3f733677bb80 None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=138ede2854894a3b995435980965911c) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:31.691358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dface367-dafe-4c34-bd14-3f733677bb80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:31.692845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dface367-dafe-4c34-bd14-3f733677bb80 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:31.707314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 264/541] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:31 2022] GET /identity/v3/role_assignments?user.id=138ede2854894a3b995435980965911c&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:31.763880 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc07e9-de3f-44cd-ae43-9bd0c470778d None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:31.764217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc07e9-de3f-44cd-ae43-9bd0c470778d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:31.764451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc07e9-de3f-44cd-ae43-9bd0c470778d None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:31.766137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 278/542] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:31.771980 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147fc55-aa22-4c79-b45d-90e1570f164f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:31.772417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147fc55-aa22-4c79-b45d-90e1570f164f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:31.772745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147fc55-aa22-4c79-b45d-90e1570f164f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:31.798258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6147fc55-aa22-4c79-b45d-90e1570f164f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:31.799506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6147fc55-aa22-4c79-b45d-90e1570f164f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:31.823194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:31.879901 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 265/543] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:31.923528 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc07e9-de3f-44cd-ae43-9bd0c470778d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:32.083699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc07e9-de3f-44cd-ae43-9bd0c470778d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:32.086126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:32.128795 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851b92d-474d-4065-98e7-e1bbcec24f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:32.129452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851b92d-474d-4065-98e7-e1bbcec24f28 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:32.129734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851b92d-474d-4065-98e7-e1bbcec24f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:32.130008 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851b92d-474d-4065-98e7-e1bbcec24f28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:32.147964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851b92d-474d-4065-98e7-e1bbcec24f28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9851b92d-474d-4065-98e7-e1bbcec24f28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:32.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:32.154544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:32.155984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 279/544] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:32.161384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147fc55-aa22-4c79-b45d-90e1570f164f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:32.409645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147fc55-aa22-4c79-b45d-90e1570f164f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:32.500371 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe29793-d8bd-4e9a-ad0b-aa7e0fe278fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:32.501422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe29793-d8bd-4e9a-ad0b-aa7e0fe278fe None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:32.501945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe29793-d8bd-4e9a-ad0b-aa7e0fe278fe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:32.502442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe29793-d8bd-4e9a-ad0b-aa7e0fe278fe None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:32.503424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe29793-d8bd-4e9a-ad0b-aa7e0fe278fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:32.503931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe29793-d8bd-4e9a-ad0b-aa7e0fe278fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:32.505578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe29793-d8bd-4e9a-ad0b-aa7e0fe278fe None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:32.520744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 266/545] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:32.528120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851b92d-474d-4065-98e7-e1bbcec24f28 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:32.694725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851b92d-474d-4065-98e7-e1bbcec24f28 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:32.745016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a63b9b-2558-46ff-a7bb-b099c38faf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:32.745721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a63b9b-2558-46ff-a7bb-b099c38faf18 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:32.746017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a63b9b-2558-46ff-a7bb-b099c38faf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:32.746296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a63b9b-2558-46ff-a7bb-b099c38faf18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:32.760535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a63b9b-2558-46ff-a7bb-b099c38faf18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:32.762335 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a63b9b-2558-46ff-a7bb-b099c38faf18 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-64a63b9b-2558-46ff-a7bb-b099c38faf18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:32.780776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:32.781675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:32.782508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:32.782508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:32.782508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:32.782508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:32.782508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 280/546] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:32.787505 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe29793-d8bd-4e9a-ad0b-aa7e0fe278fe None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:32.807465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:32.872970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bce267-2640-4162-b7c0-e9149bbe5a61 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:32.873265 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bce267-2640-4162-b7c0-e9149bbe5a61 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:32.873527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bce267-2640-4162-b7c0-e9149bbe5a61 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:32.875127 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 281/547] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:32.880740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2a077-e1ae-4c38-900c-3fb137c642b4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:32.881028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2a077-e1ae-4c38-900c-3fb137c642b4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:32.881308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2a077-e1ae-4c38-900c-3fb137c642b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:32.899174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe29793-d8bd-4e9a-ad0b-aa7e0fe278fe None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:32.909063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-70d2a077-e1ae-4c38-900c-3fb137c642b4 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:32.910263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-70d2a077-e1ae-4c38-900c-3fb137c642b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:32.931450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:32.942077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efea8c0-6dde-44de-9a72-e5ef1d6d54eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:32.943008 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efea8c0-6dde-44de-9a72-e5ef1d6d54eb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:32.943516 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efea8c0-6dde-44de-9a72-e5ef1d6d54eb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:32.944007 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efea8c0-6dde-44de-9a72-e5ef1d6d54eb None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:32.951276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efea8c0-6dde-44de-9a72-e5ef1d6d54eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:32.951900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efea8c0-6dde-44de-9a72-e5ef1d6d54eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:32.953551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efea8c0-6dde-44de-9a72-e5ef1d6d54eb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:32.971027 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 267/548] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:32.976852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efea8c0-6dde-44de-9a72-e5ef1d6d54eb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:33.064491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 282/549] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:33.080747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac842f4-cf6f-4958-a903-f6f28f96a224 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:33.081058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac842f4-cf6f-4958-a903-f6f28f96a224 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:33.081305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac842f4-cf6f-4958-a903-f6f28f96a224 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:33.082918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 283/550] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:24:33.088233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d2284-7809-430f-9ee2-d45122b20bde None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:33.088546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d2284-7809-430f-9ee2-d45122b20bde None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:33.088938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d2284-7809-430f-9ee2-d45122b20bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:33.097323 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:33.107430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-406d2284-7809-430f-9ee2-d45122b20bde None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:33.108735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-406d2284-7809-430f-9ee2-d45122b20bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:33.121539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efea8c0-6dde-44de-9a72-e5ef1d6d54eb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:33.163196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb4dfa-82fa-486a-b5b2-3a380093d866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:33.164096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb4dfa-82fa-486a-b5b2-3a380093d866 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:33.164381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb4dfa-82fa-486a-b5b2-3a380093d866 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:33.164661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb4dfa-82fa-486a-b5b2-3a380093d866 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:33.174146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb4dfa-82fa-486a-b5b2-3a380093d866 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:33.176311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb4dfa-82fa-486a-b5b2-3a380093d866 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:33.184118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 284/551] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:33.185032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d2284-7809-430f-9ee2-d45122b20bde None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fedb4dfa-82fa-486a-b5b2-3a380093d866 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:33.202487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:33.203928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:33.204977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:33.204977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:33.204977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:33.204977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:33.204977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:33.204977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:33.204977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:33.204977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:33.204977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:24:33.204977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:33.207259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 268/552] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:33.212119 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb4dfa-82fa-486a-b5b2-3a380093d866 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:33.280064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:33.307258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d2284-7809-430f-9ee2-d45122b20bde None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:33.346162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd6c91-db00-4cf6-a81f-71a431e85d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:33.347056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd6c91-db00-4cf6-a81f-71a431e85d0c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:33.347517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd6c91-db00-4cf6-a81f-71a431e85d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:33.347896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd6c91-db00-4cf6-a81f-71a431e85d0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:33.349178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb4dfa-82fa-486a-b5b2-3a380093d866 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:33.358998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd6c91-db00-4cf6-a81f-71a431e85d0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a0fd6c91-db00-4cf6-a81f-71a431e85d0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:33.364420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:33.365425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:33.366377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:33.366377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:33.366377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:33.366377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:33.366377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:33.366377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:33.366377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:33.366377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:33.366377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:33.366377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:33.366377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:33.366377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:33.371436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 285/553] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:33.372311 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd6c91-db00-4cf6-a81f-71a431e85d0c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:33.388899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b6ca0-17a3-481f-9e2c-6b52005613fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:33.389544 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b6ca0-17a3-481f-9e2c-6b52005613fb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:33.390072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b6ca0-17a3-481f-9e2c-6b52005613fb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:33.390348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b6ca0-17a3-481f-9e2c-6b52005613fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:33.394068 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:33.404808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b6ca0-17a3-481f-9e2c-6b52005613fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e08b6ca0-17a3-481f-9e2c-6b52005613fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:33.412273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:33.414163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:33.415283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:33.415283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:33.415283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:33.415283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:33.415283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:33.415283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:33.415283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:33.415283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:33.415283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:33.415283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:33.415283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:33.415283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:33.421309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 269/554] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:33.421309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b6ca0-17a3-481f-9e2c-6b52005613fb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:33.494295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd6c91-db00-4cf6-a81f-71a431e85d0c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:33.531908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22184fa-794d-40fc-a0ed-b118e9352c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:33.532891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22184fa-794d-40fc-a0ed-b118e9352c88 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:33.533281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22184fa-794d-40fc-a0ed-b118e9352c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:33.533943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22184fa-794d-40fc-a0ed-b118e9352c88 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:33.534848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22184fa-794d-40fc-a0ed-b118e9352c88 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:33.535292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22184fa-794d-40fc-a0ed-b118e9352c88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:33.537295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22184fa-794d-40fc-a0ed-b118e9352c88 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:33.542407 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:33.552895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b6ca0-17a3-481f-9e2c-6b52005613fb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:33.563370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 286/555] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Aug 1 07:24:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:33.564482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22184fa-794d-40fc-a0ed-b118e9352c88 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:33.595897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf463d5-d919-430e-ab82-98b32dca8503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:33.596828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf463d5-d919-430e-ab82-98b32dca8503 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:33.597118 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf463d5-d919-430e-ab82-98b32dca8503 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:33.597396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf463d5-d919-430e-ab82-98b32dca8503 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:33.598367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf463d5-d919-430e-ab82-98b32dca8503 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:33.598674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf463d5-d919-430e-ab82-98b32dca8503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:33.600699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf463d5-d919-430e-ab82-98b32dca8503 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:33.611362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 270/556] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:33.612212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf463d5-d919-430e-ab82-98b32dca8503 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:33.683543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22184fa-794d-40fc-a0ed-b118e9352c88 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:33.695623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:33.726946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca6978-e522-40eb-b62c-4f3e5b530854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:33.727791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca6978-e522-40eb-b62c-4f3e5b530854 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:33.728361 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca6978-e522-40eb-b62c-4f3e5b530854 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:33.728683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca6978-e522-40eb-b62c-4f3e5b530854 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:33.729720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca6978-e522-40eb-b62c-4f3e5b530854 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:33.730159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca6978-e522-40eb-b62c-4f3e5b530854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:33.731847 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca6978-e522-40eb-b62c-4f3e5b530854 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:33.747605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf463d5-d919-430e-ab82-98b32dca8503 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:33.747781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 287/557] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:33.747781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca6978-e522-40eb-b62c-4f3e5b530854 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:33.786311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ed944-07c2-43c3-a174-f04f9ed18d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:33.786930 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ed944-07c2-43c3-a174-f04f9ed18d76 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:33.788009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ed944-07c2-43c3-a174-f04f9ed18d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:33.788292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ed944-07c2-43c3-a174-f04f9ed18d76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:33.811217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:33.862515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca6978-e522-40eb-b62c-4f3e5b530854 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:33.873994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ed944-07c2-43c3-a174-f04f9ed18d76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:33.876086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ed944-07c2-43c3-a174-f04f9ed18d76 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:33.901451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75707335-d84c-4019-9e23-c3b23d6d6c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:33.902716 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75707335-d84c-4019-9e23-c3b23d6d6c8d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:33.903011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75707335-d84c-4019-9e23-c3b23d6d6c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:33.903599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75707335-d84c-4019-9e23-c3b23d6d6c8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:33.919750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75707335-d84c-4019-9e23-c3b23d6d6c8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:33.921215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75707335-d84c-4019-9e23-c3b23d6d6c8d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-75707335-d84c-4019-9e23-c3b23d6d6c8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:33.941858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:33.942800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:33.943728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:33.943728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:33.943728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:33.943728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:33.949733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 288/558] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:33.949733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75707335-d84c-4019-9e23-c3b23d6d6c8d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:33.958291 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:34.015337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 271/559] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:34.016107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ed944-07c2-43c3-a174-f04f9ed18d76 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:34.073123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75707335-d84c-4019-9e23-c3b23d6d6c8d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:34.081617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:34.109039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cecdac-43ea-4e30-b9cc-120fff200057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:34.109679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cecdac-43ea-4e30-b9cc-120fff200057 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.110000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cecdac-43ea-4e30-b9cc-120fff200057 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.110285 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cecdac-43ea-4e30-b9cc-120fff200057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.113806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ed944-07c2-43c3-a174-f04f9ed18d76 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:34.120797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cecdac-43ea-4e30-b9cc-120fff200057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:34.122266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cecdac-43ea-4e30-b9cc-120fff200057 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-18cecdac-43ea-4e30-b9cc-120fff200057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:34.136581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:34.137693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:34.138616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:34.138616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:34.138616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:34.138616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:34.138616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 289/560] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:34.139208 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459ca45-d80e-447a-962c-ba2aa4e2d0f6 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.139496 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459ca45-d80e-447a-962c-ba2aa4e2d0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.139772 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459ca45-d80e-447a-962c-ba2aa4e2d0f6 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.141013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 290/561] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:34.141990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbae91c-6a90-443e-8e1b-3a5dd5d7873b None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.142273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbae91c-6a90-443e-8e1b-3a5dd5d7873b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.142549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbae91c-6a90-443e-8e1b-3a5dd5d7873b None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.143751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 291/562] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:34.144433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbae91c-6a90-443e-8e1b-3a5dd5d7873b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:34.157490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f848a-fac9-4147-af91-df86877ae5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:34.158110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f848a-fac9-4147-af91-df86877ae5fe None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.158397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f848a-fac9-4147-af91-df86877ae5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.158678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f848a-fac9-4147-af91-df86877ae5fe None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.159398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f848a-fac9-4147-af91-df86877ae5fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:34.159714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f848a-fac9-4147-af91-df86877ae5fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:34.161073 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f848a-fac9-4147-af91-df86877ae5fe None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:34.177425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 272/563] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:34.177425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f848a-fac9-4147-af91-df86877ae5fe None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:34.196302 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:34.250074 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbae91c-6a90-443e-8e1b-3a5dd5d7873b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:34.277553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f848a-fac9-4147-af91-df86877ae5fe None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:34.285112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e5b4d-5272-4929-a922-df0ec8441dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:34.285749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e5b4d-5272-4929-a922-df0ec8441dfa None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.286036 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e5b4d-5272-4929-a922-df0ec8441dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.286317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e5b4d-5272-4929-a922-df0ec8441dfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.296594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e5b4d-5272-4929-a922-df0ec8441dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:34.298037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e5b4d-5272-4929-a922-df0ec8441dfa None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:34.311002 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a06f60-1303-4914-87c7-51420c8223d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:34.311665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a06f60-1303-4914-87c7-51420c8223d3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.311948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a06f60-1303-4914-87c7-51420c8223d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.312229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a06f60-1303-4914-87c7-51420c8223d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.312902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a06f60-1303-4914-87c7-51420c8223d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:34.313207 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a06f60-1303-4914-87c7-51420c8223d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-2f3e5b4d-5272-4929-a922-df0ec8441dfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:34.313391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:34.314234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:34.315381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:34.315381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:34.315381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:34.315381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:34.315601 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a06f60-1303-4914-87c7-51420c8223d3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:34.315685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 292/564] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:34.315967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d475f2-d948-4e6d-9a29-f62f0870a48d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.316249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d475f2-d948-4e6d-9a29-f62f0870a48d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.316524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d475f2-d948-4e6d-9a29-f62f0870a48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.330520 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 273/565] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:34.331753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cef52a-f238-4674-b41c-7c87d5626825 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.332038 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cef52a-f238-4674-b41c-7c87d5626825 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.332312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cef52a-f238-4674-b41c-7c87d5626825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.334785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e4d475f2-d948-4e6d-9a29-f62f0870a48d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:34.335992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e4d475f2-d948-4e6d-9a29-f62f0870a48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:34.345209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:34.350252 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d1cef52a-f238-4674-b41c-7c87d5626825 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:34.351444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d1cef52a-f238-4674-b41c-7c87d5626825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:34.405765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 293/566] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:34.406681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d475f2-d948-4e6d-9a29-f62f0870a48d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:34.419443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 274/567] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:34.420648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cef52a-f238-4674-b41c-7c87d5626825 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:34.443761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:34.505923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d475f2-d948-4e6d-9a29-f62f0870a48d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:34.531615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cef52a-f238-4674-b41c-7c87d5626825 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:34.539522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44207209-ed9f-4d98-a211-e3e2e327d8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:34.540167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207209-ed9f-4d98-a211-e3e2e327d8ba None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.540456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207209-ed9f-4d98-a211-e3e2e327d8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.541308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207209-ed9f-4d98-a211-e3e2e327d8ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.548316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44207209-ed9f-4d98-a211-e3e2e327d8ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:34.549760 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44207209-ed9f-4d98-a211-e3e2e327d8ba None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-44207209-ed9f-4d98-a211-e3e2e327d8ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:34.561310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:34.562388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:34.563359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:34.563359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:34.563359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:34.563359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:34.563359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:34.563359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:34.563359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:34.563359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:34.563359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:24:34.563359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:34.563359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 294/568] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Aug 1 07:24:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:34.563359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44207209-ed9f-4d98-a211-e3e2e327d8ba None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:34.565971 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:34.573979 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce94fae-5f0b-4de1-ab56-c8488afda4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:34.574606 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce94fae-5f0b-4de1-ab56-c8488afda4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.574892 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce94fae-5f0b-4de1-ab56-c8488afda4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.575202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce94fae-5f0b-4de1-ab56-c8488afda4e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.575900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce94fae-5f0b-4de1-ab56-c8488afda4e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:34.576209 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce94fae-5f0b-4de1-ab56-c8488afda4e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:34.577578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce94fae-5f0b-4de1-ab56-c8488afda4e7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:34.596633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 275/569] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:34.597454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce94fae-5f0b-4de1-ab56-c8488afda4e7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:34.663208 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:34.674625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44207209-ed9f-4d98-a211-e3e2e327d8ba None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:34.696599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce94fae-5f0b-4de1-ab56-c8488afda4e7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:34.708236 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7180688-4fb4-4b0a-b918-f78e942614ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:34.708872 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7180688-4fb4-4b0a-b918-f78e942614ac None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.709165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7180688-4fb4-4b0a-b918-f78e942614ac None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.709444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7180688-4fb4-4b0a-b918-f78e942614ac None admin] PATH_INFO: `/v3/users/glance` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.717773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7180688-4fb4-4b0a-b918-f78e942614ac None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:34.719254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7180688-4fb4-4b0a-b918-f78e942614ac None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:34.730636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa782f00-cc2e-428d-8974-3baadd4be642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-c7180688-4fb4-4b0a-b918-f78e942614ac None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:34.730863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:34.731810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:34.732744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:34.732744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:34.732744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:34.732744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:34.732744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:34.732744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:34.732744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:34.732744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:34.732744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 07:24:34.732744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:34.733259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa782f00-cc2e-428d-8974-3baadd4be642 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.733259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa782f00-cc2e-428d-8974-3baadd4be642 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.733259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa782f00-cc2e-428d-8974-3baadd4be642 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.733451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 295/570] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:34.733451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7180688-4fb4-4b0a-b918-f78e942614ac None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:34.745184 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa782f00-cc2e-428d-8974-3baadd4be642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fa782f00-cc2e-428d-8974-3baadd4be642 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:34.750472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:34.751427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:34.752281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 276/571] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:34.752972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa782f00-cc2e-428d-8974-3baadd4be642 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:34.797391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:34.836209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7180688-4fb4-4b0a-b918-f78e942614ac None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:34.852762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa782f00-cc2e-428d-8974-3baadd4be642 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:34.870607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59599e00-389d-485e-8c67-51372dba43ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:34.871266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59599e00-389d-485e-8c67-51372dba43ad None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.871548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59599e00-389d-485e-8c67-51372dba43ad None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.871821 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59599e00-389d-485e-8c67-51372dba43ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.878734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:34.882486 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59599e00-389d-485e-8c67-51372dba43ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:34.887258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b0827-d5d7-4cc1-bfff-e450fc64974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:34.887870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b0827-d5d7-4cc1-bfff-e450fc64974a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:34.888158 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b0827-d5d7-4cc1-bfff-e450fc64974a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-59599e00-389d-485e-8c67-51372dba43ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:34.888280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:34.889425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:34.890298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:34.890298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:34.890298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:34.890298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:34.890298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:34.890298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:34.890298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:34.890298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:34.890298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:34.890298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:34.890298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:34.890298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:34.890978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b0827-d5d7-4cc1-bfff-e450fc64974a None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:34.890978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b0827-d5d7-4cc1-bfff-e450fc64974a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:34.890978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b0827-d5d7-4cc1-bfff-e450fc64974a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:34.891189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 296/572] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:34.891189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59599e00-389d-485e-8c67-51372dba43ad None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:34.891311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b0827-d5d7-4cc1-bfff-e450fc64974a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:34.909014 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 277/573] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Aug 1 07:24:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:34.909822 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b0827-d5d7-4cc1-bfff-e450fc64974a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:34.996338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59599e00-389d-485e-8c67-51372dba43ad None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:35.009336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b0827-d5d7-4cc1-bfff-e450fc64974a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:35.020436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:35.033029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b1968-1e68-4e88-a681-31b7e3777da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:35.033671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b1968-1e68-4e88-a681-31b7e3777da4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:35.033954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b1968-1e68-4e88-a681-31b7e3777da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:35.034236 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b1968-1e68-4e88-a681-31b7e3777da4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:35.041331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b1968-1e68-4e88-a681-31b7e3777da4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:35.042757 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b1968-1e68-4e88-a681-31b7e3777da4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:35.044273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d8f86-024a-44ee-ba4f-c1f66a2baf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:35.044890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d8f86-024a-44ee-ba4f-c1f66a2baf83 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:35.045173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d8f86-024a-44ee-ba4f-c1f66a2baf83 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:35.045456 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d8f86-024a-44ee-ba4f-c1f66a2baf83 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:35.046238 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d8f86-024a-44ee-ba4f-c1f66a2baf83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:35.046548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d8f86-024a-44ee-ba4f-c1f66a2baf83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:35.048037 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d8f86-024a-44ee-ba4f-c1f66a2baf83 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-d97b1968-1e68-4e88-a681-31b7e3777da4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:35.054120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:35.055211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:35.056125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:35.056125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:35.056125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:35.056125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:35.056125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:35.056125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:35.056125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:35.056125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:35.056125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 297/574] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:35.057532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b1968-1e68-4e88-a681-31b7e3777da4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:35.068193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 278/575] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Aug 1 07:24:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:35.069012 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d8f86-024a-44ee-ba4f-c1f66a2baf83 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:35.113285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:35.156964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b1968-1e68-4e88-a681-31b7e3777da4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:35.167217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d8f86-024a-44ee-ba4f-c1f66a2baf83 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:35.190620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61e36d-779c-414c-99e5-b89853610dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:35.191261 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61e36d-779c-414c-99e5-b89853610dfd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:35.191547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61e36d-779c-414c-99e5-b89853610dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:35.191822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61e36d-779c-414c-99e5-b89853610dfd None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:35.192529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61e36d-779c-414c-99e5-b89853610dfd None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:35.192838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61e36d-779c-414c-99e5-b89853610dfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:35.194204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61e36d-779c-414c-99e5-b89853610dfd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:35.200252 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64683d-9cc1-4e65-8ac2-bc1dccba46ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:35.200864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64683d-9cc1-4e65-8ac2-bc1dccba46ab None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:35.201149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64683d-9cc1-4e65-8ac2-bc1dccba46ab None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:35.201429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64683d-9cc1-4e65-8ac2-bc1dccba46ab None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:35.202001 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 298/576] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:35.202124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64683d-9cc1-4e65-8ac2-bc1dccba46ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:35.202441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64683d-9cc1-4e65-8ac2-bc1dccba46ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:35.202783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61e36d-779c-414c-99e5-b89853610dfd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:35.203828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64683d-9cc1-4e65-8ac2-bc1dccba46ab None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:35.213384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 279/577] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:35.214181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64683d-9cc1-4e65-8ac2-bc1dccba46ab None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:35.249742 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:35.305145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61e36d-779c-414c-99e5-b89853610dfd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:35.316579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64683d-9cc1-4e65-8ac2-bc1dccba46ab None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:35.323872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:35.339094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98e5d1-78dd-49aa-8fd9-a05a6123d1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:35.339736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98e5d1-78dd-49aa-8fd9-a05a6123d1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:35.340018 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98e5d1-78dd-49aa-8fd9-a05a6123d1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:35.340291 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98e5d1-78dd-49aa-8fd9-a05a6123d1b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:35.350462 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98e5d1-78dd-49aa-8fd9-a05a6123d1b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:35.350582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f90f06-6be4-49c4-8ccc-59710f888a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:35.351170 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f90f06-6be4-49c4-8ccc-59710f888a19 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:35.351461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f90f06-6be4-49c4-8ccc-59710f888a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:35.351745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f90f06-6be4-49c4-8ccc-59710f888a19 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:35.351926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98e5d1-78dd-49aa-8fd9-a05a6123d1b4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:35.352490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f90f06-6be4-49c4-8ccc-59710f888a19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:35.352802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f90f06-6be4-49c4-8ccc-59710f888a19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:35.354415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f90f06-6be4-49c4-8ccc-59710f888a19 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-6f98e5d1-78dd-49aa-8fd9-a05a6123d1b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:35.365625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:35.366644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:35.367766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:35.367766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:35.367766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:35.367766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:35.368037 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 280/578] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:35.368037 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f90f06-6be4-49c4-8ccc-59710f888a19 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:35.371986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 299/579] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:35.372785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98e5d1-78dd-49aa-8fd9-a05a6123d1b4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:35.465931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:35.467496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f90f06-6be4-49c4-8ccc-59710f888a19 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:35.472168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98e5d1-78dd-49aa-8fd9-a05a6123d1b4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:35.501408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1411fd5-5dd1-4a01-b6c8-af517d8b71a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:35.502043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1411fd5-5dd1-4a01-b6c8-af517d8b71a2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:35.502323 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1411fd5-5dd1-4a01-b6c8-af517d8b71a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:35.502605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1411fd5-5dd1-4a01-b6c8-af517d8b71a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:35.507216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc08e-d15c-406e-a698-5a29f878af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:35.507843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc08e-d15c-406e-a698-5a29f878af2a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:35.508116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc08e-d15c-406e-a698-5a29f878af2a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:35.508393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc08e-d15c-406e-a698-5a29f878af2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:35.512765 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1411fd5-5dd1-4a01-b6c8-af517d8b71a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:35.514234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1411fd5-5dd1-4a01-b6c8-af517d8b71a2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:35.519182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bc08e-d15c-406e-a698-5a29f878af2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:35.520643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bc08e-d15c-406e-a698-5a29f878af2a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e1411fd5-5dd1-4a01-b6c8-af517d8b71a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:35.530410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:35.531342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:35.532235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:35.532235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:35.532235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:35.532235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:35.532235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 281/580] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:35.532630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1411fd5-5dd1-4a01-b6c8-af517d8b71a2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9d4bc08e-d15c-406e-a698-5a29f878af2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:35.535086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:35.536221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:35.537469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:35.537469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:35.537469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:35.537469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:35.539402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 300/581] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:35.540003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc08e-d15c-406e-a698-5a29f878af2a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:35.553861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:35.633441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1411fd5-5dd1-4a01-b6c8-af517d8b71a2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:35.639028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc08e-d15c-406e-a698-5a29f878af2a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:35.666374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedca358-0946-40b0-a4cb-54f97b02284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:35.667004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedca358-0946-40b0-a4cb-54f97b02284e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:35.667312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedca358-0946-40b0-a4cb-54f97b02284e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:35.667589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedca358-0946-40b0-a4cb-54f97b02284e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:35.672501 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb40d2d-1c88-4dc8-b75c-ff57dabf8186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:35.673108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb40d2d-1c88-4dc8-b75c-ff57dabf8186 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:35.673395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb40d2d-1c88-4dc8-b75c-ff57dabf8186 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:35.673677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb40d2d-1c88-4dc8-b75c-ff57dabf8186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:35.674462 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb40d2d-1c88-4dc8-b75c-ff57dabf8186 None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=2ba2655c2eef408498c0298d432263f6) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:35.674778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb40d2d-1c88-4dc8-b75c-ff57dabf8186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:35.676249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb40d2d-1c88-4dc8-b75c-ff57dabf8186 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:35.677838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedca358-0946-40b0-a4cb-54f97b02284e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:35.679301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedca358-0946-40b0-a4cb-54f97b02284e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:35.681597 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 301/582] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:35 2022] GET /identity/v3/role_assignments?user.id=2ba2655c2eef408498c0298d432263f6&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:35.682373 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb40d2d-1c88-4dc8-b75c-ff57dabf8186 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:35.700341 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cedca358-0946-40b0-a4cb-54f97b02284e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:35.705340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:35.706295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:35.707212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:35.707212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:35.707212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:35.707212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:35.707212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 282/583] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:35.707485 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedca358-0946-40b0-a4cb-54f97b02284e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:35.787358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:35.814541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb40d2d-1c88-4dc8-b75c-ff57dabf8186 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:35.820887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedca358-0946-40b0-a4cb-54f97b02284e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:35.856007 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49905f-069a-4757-8820-fec7d9ec8c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:35.856625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49905f-069a-4757-8820-fec7d9ec8c1c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:35.856907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49905f-069a-4757-8820-fec7d9ec8c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:35.857180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49905f-069a-4757-8820-fec7d9ec8c1c None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:35.857857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49905f-069a-4757-8820-fec7d9ec8c1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:35.858169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49905f-069a-4757-8820-fec7d9ec8c1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:35.859630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49905f-069a-4757-8820-fec7d9ec8c1c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:35.861420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae22f-ec5d-4409-a5c7-36100409515b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:35.862318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae22f-ec5d-4409-a5c7-36100409515b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:35.862425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae22f-ec5d-4409-a5c7-36100409515b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:35.862692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae22f-ec5d-4409-a5c7-36100409515b None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:35.863413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cae22f-ec5d-4409-a5c7-36100409515b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:35.863718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cae22f-ec5d-4409-a5c7-36100409515b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:35.865133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cae22f-ec5d-4409-a5c7-36100409515b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:35.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 302/584] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:35.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49905f-069a-4757-8820-fec7d9ec8c1c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:35.887289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 283/585] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:35.887289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae22f-ec5d-4409-a5c7-36100409515b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:35.952841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:35.988428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49905f-069a-4757-8820-fec7d9ec8c1c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:36.003136 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae22f-ec5d-4409-a5c7-36100409515b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:36.027624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebab922-7cac-4d27-aad0-40e706391234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:36.028371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebab922-7cac-4d27-aad0-40e706391234 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:36.028659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebab922-7cac-4d27-aad0-40e706391234 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:36.029091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebab922-7cac-4d27-aad0-40e706391234 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:36.030116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebab922-7cac-4d27-aad0-40e706391234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:36.030703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebab922-7cac-4d27-aad0-40e706391234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:36.032278 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebab922-7cac-4d27-aad0-40e706391234 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:36.034650 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:36.041616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326547bb-f93f-48f1-b306-c4d0a263afa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:36.042635 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326547bb-f93f-48f1-b306-c4d0a263afa9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:36.042936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326547bb-f93f-48f1-b306-c4d0a263afa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:36.043247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326547bb-f93f-48f1-b306-c4d0a263afa9 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:36.043935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326547bb-f93f-48f1-b306-c4d0a263afa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:36.044241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326547bb-f93f-48f1-b306-c4d0a263afa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:36.045596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326547bb-f93f-48f1-b306-c4d0a263afa9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:36.047348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 303/586] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:36.048813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebab922-7cac-4d27-aad0-40e706391234 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:36.063648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 284/587] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:36.063648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326547bb-f93f-48f1-b306-c4d0a263afa9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:36.159649 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebab922-7cac-4d27-aad0-40e706391234 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:36.171185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326547bb-f93f-48f1-b306-c4d0a263afa9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:36.190290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:36.198089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd558e-cc06-4ab2-80bb-4f954d6ea1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:36.198732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd558e-cc06-4ab2-80bb-4f954d6ea1eb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:36.199012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd558e-cc06-4ab2-80bb-4f954d6ea1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:36.199320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd558e-cc06-4ab2-80bb-4f954d6ea1eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:36.206465 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd558e-cc06-4ab2-80bb-4f954d6ea1eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:36.206804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98813492-f592-4320-9a34-8beb241a72a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:36.207477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98813492-f592-4320-9a34-8beb241a72a7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:36.207762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98813492-f592-4320-9a34-8beb241a72a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:36.207967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd558e-cc06-4ab2-80bb-4f954d6ea1eb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:36.208064 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98813492-f592-4320-9a34-8beb241a72a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:36.214837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98813492-f592-4320-9a34-8beb241a72a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:36.216327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98813492-f592-4320-9a34-8beb241a72a7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9edd558e-cc06-4ab2-80bb-4f954d6ea1eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:36.220141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:36.221003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:36.222518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:36.222518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:36.222518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:36.222518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:36.222518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:36.222518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:36.222518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:36.222518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:36.222518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 304/588] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:36.222518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd558e-cc06-4ab2-80bb-4f954d6ea1eb None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-98813492-f592-4320-9a34-8beb241a72a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:36.233071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:36.233961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:36.234866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:36.234866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:36.234866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:36.234866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:36.234866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:36.234866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:36.234866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:36.234866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:36.235852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 285/589] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:36.235852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98813492-f592-4320-9a34-8beb241a72a7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:36.288265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:36.346125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98813492-f592-4320-9a34-8beb241a72a7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:36.348194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd558e-cc06-4ab2-80bb-4f954d6ea1eb None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:36.380127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf15c4e-8a6d-40fa-8f8a-23e4c7463483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:36.380766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf15c4e-8a6d-40fa-8f8a-23e4c7463483 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:36.381054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf15c4e-8a6d-40fa-8f8a-23e4c7463483 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:36.381335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf15c4e-8a6d-40fa-8f8a-23e4c7463483 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:36.385603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678a32c-28fc-414a-93b0-f2c683213860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:36.386421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678a32c-28fc-414a-93b0-f2c683213860 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:36.386845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678a32c-28fc-414a-93b0-f2c683213860 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:36.387302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678a32c-28fc-414a-93b0-f2c683213860 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:36.389632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf15c4e-8a6d-40fa-8f8a-23e4c7463483 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:36.391092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf15c4e-8a6d-40fa-8f8a-23e4c7463483 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:36.396128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678a32c-28fc-414a-93b0-f2c683213860 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:36.397713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678a32c-28fc-414a-93b0-f2c683213860 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-acf15c4e-8a6d-40fa-8f8a-23e4c7463483 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:36.402405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:36.403465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:36.409080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:36.409080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:36.409080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:36.409080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:36.409080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:36.409080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:36.409080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:36.409080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:36.409080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:24:36.409080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:36.409080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 286/590] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:36.409080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf15c4e-8a6d-40fa-8f8a-23e4c7463483 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-d678a32c-28fc-414a-93b0-f2c683213860 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:36.411875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:36.412752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:36.413671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:36.413671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:36.413671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:36.413671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:36.413671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:36.413671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:36.413671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:36.413671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:36.413671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 07:24:36.413671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:36.413671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 305/591] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:36.415264 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678a32c-28fc-414a-93b0-f2c683213860 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:36.471946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:36.517651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf15c4e-8a6d-40fa-8f8a-23e4c7463483 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:36.527086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678a32c-28fc-414a-93b0-f2c683213860 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:36.527975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:36.560453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9eeb6-abe2-4f8d-a18c-d5ae643efdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:36.561260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9eeb6-abe2-4f8d-a18c-d5ae643efdc2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:36.561630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9eeb6-abe2-4f8d-a18c-d5ae643efdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:36.561973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9eeb6-abe2-4f8d-a18c-d5ae643efdc2 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:36.562907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9eeb6-abe2-4f8d-a18c-d5ae643efdc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:36.563244 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9eeb6-abe2-4f8d-a18c-d5ae643efdc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:36.564872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9eeb6-abe2-4f8d-a18c-d5ae643efdc2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:36.570905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b6c05-a67b-43dc-abcb-b76faf328ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:36.571587 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b6c05-a67b-43dc-abcb-b76faf328ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:36.571918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b6c05-a67b-43dc-abcb-b76faf328ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:36.572201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b6c05-a67b-43dc-abcb-b76faf328ad9 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:36.572935 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b6c05-a67b-43dc-abcb-b76faf328ad9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:36.573642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b6c05-a67b-43dc-abcb-b76faf328ad9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:36.574664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b6c05-a67b-43dc-abcb-b76faf328ad9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:36.585488 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 287/592] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:36.585488 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9eeb6-abe2-4f8d-a18c-d5ae643efdc2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:36.593111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 306/593] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:36.593859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b6c05-a67b-43dc-abcb-b76faf328ad9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:36.700873 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9eeb6-abe2-4f8d-a18c-d5ae643efdc2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:36.709356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b6c05-a67b-43dc-abcb-b76faf328ad9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:36.723832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:36.739654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6aba3-d955-4f33-bdb0-66494b62bd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:36.740287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6aba3-d955-4f33-bdb0-66494b62bd9f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:36.740563 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6aba3-d955-4f33-bdb0-66494b62bd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:36.740844 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6aba3-d955-4f33-bdb0-66494b62bd9f None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:36.741636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6aba3-d955-4f33-bdb0-66494b62bd9f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:36.741950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6aba3-d955-4f33-bdb0-66494b62bd9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:36.743396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6aba3-d955-4f33-bdb0-66494b62bd9f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:36.752347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6fcc3-c656-4362-83c3-f7d76a60dcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:36.753364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6fcc3-c656-4362-83c3-f7d76a60dcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:36.753875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6fcc3-c656-4362-83c3-f7d76a60dcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:36.754214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6fcc3-c656-4362-83c3-f7d76a60dcf6 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:36.755148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6fcc3-c656-4362-83c3-f7d76a60dcf6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:36.755467 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6fcc3-c656-4362-83c3-f7d76a60dcf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:36.756891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6fcc3-c656-4362-83c3-f7d76a60dcf6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:36.767335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 288/594] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Aug 1 07:24:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:36.767335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6aba3-d955-4f33-bdb0-66494b62bd9f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:36.786312 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 307/595] 10.222.0.40 () {56 vars in 1139 bytes} [Mon Aug 1 07:24:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:36.786312 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6fcc3-c656-4362-83c3-f7d76a60dcf6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:36.802807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:36.881311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6aba3-d955-4f33-bdb0-66494b62bd9f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:36.898688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6fcc3-c656-4362-83c3-f7d76a60dcf6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:36.917771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae41a0-153c-4aca-b0ad-951a4755d9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:36.918446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae41a0-153c-4aca-b0ad-951a4755d9e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:36.918727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae41a0-153c-4aca-b0ad-951a4755d9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:36.919006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae41a0-153c-4aca-b0ad-951a4755d9e1 None admin] PATH_INFO: `/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/abdcd6d09fee45c696449641ea36eae3/roles/74131e9234dc48a2b55332d15e8c6462` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:36.934991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae41a0-153c-4aca-b0ad-951a4755d9e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ddf7d6f2424a85a741d017497b49c9, user_id=abdcd6d09fee45c696449641ea36eae3, role_id=74131e9234dc48a2b55332d15e8c6462)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:36.936659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae41a0-153c-4aca-b0ad-951a4755d9e1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:36.936913 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c121cd3-d9e2-454a-a22c-38e5903a3e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:36.937582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c121cd3-d9e2-454a-a22c-38e5903a3e3a None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:36.937865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c121cd3-d9e2-454a-a22c-38e5903a3e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:36.938144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c121cd3-d9e2-454a-a22c-38e5903a3e3a None admin] PATH_INFO: `/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/5f57fa108c3d478a9976f4aea5f33bb5/roles/74131e9234dc48a2b55332d15e8c6462` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:36.954397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 289/596] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:24:36 2022] PUT /identity/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/abdcd6d09fee45c696449641ea36eae3/roles/74131e9234dc48a2b55332d15e8c6462 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:24:36.955597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae41a0-153c-4aca-b0ad-951a4755d9e1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:36.957336 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c121cd3-d9e2-454a-a22c-38e5903a3e3a None admin] RBAC: Authorizing `identity:create_grant(project_id=08ddf7d6f2424a85a741d017497b49c9, user_id=5f57fa108c3d478a9976f4aea5f33bb5, role_id=74131e9234dc48a2b55332d15e8c6462)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:36.958935 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c121cd3-d9e2-454a-a22c-38e5903a3e3a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:36.991384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 308/597] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:24:36 2022] PUT /identity/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/5f57fa108c3d478a9976f4aea5f33bb5/roles/74131e9234dc48a2b55332d15e8c6462 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:24:36.992417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c121cd3-d9e2-454a-a22c-38e5903a3e3a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:37.053635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:37.098170 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:37.145542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae41a0-153c-4aca-b0ad-951a4755d9e1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:37.230047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9dc34-27cb-49ab-bd59-393a02f09cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:37.230665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9dc34-27cb-49ab-bd59-393a02f09cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:37.230889 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9dc34-27cb-49ab-bd59-393a02f09cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:37.231874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9dc34-27cb-49ab-bd59-393a02f09cd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:37.238831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c121cd3-d9e2-454a-a22c-38e5903a3e3a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:37.246163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9dc34-27cb-49ab-bd59-393a02f09cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:37.247683 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9dc34-27cb-49ab-bd59-393a02f09cd8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-14d9dc34-27cb-49ab-bd59-393a02f09cd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:37.267912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:37.268890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:37.269798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:37.269798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:37.269798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:37.269798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:37.269798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 290/598] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:37.276476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9dc34-27cb-49ab-bd59-393a02f09cd8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:37.307807 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca938413-01cc-4c3b-b277-5ebcdc238214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:37.308764 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca938413-01cc-4c3b-b277-5ebcdc238214 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:37.309198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca938413-01cc-4c3b-b277-5ebcdc238214 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:37.309624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca938413-01cc-4c3b-b277-5ebcdc238214 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:37.344917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca938413-01cc-4c3b-b277-5ebcdc238214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:37.346721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca938413-01cc-4c3b-b277-5ebcdc238214 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ca938413-01cc-4c3b-b277-5ebcdc238214 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:37.376719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:37.377810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:37.378699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:37.378699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:37.378699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:37.378699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:37.388753 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 309/599] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:37.394286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9dc34-27cb-49ab-bd59-393a02f09cd8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:37.395430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca938413-01cc-4c3b-b277-5ebcdc238214 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:37.432599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:37.453124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aadada-2047-4023-8d1c-420ac5e7a6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:37.453934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aadada-2047-4023-8d1c-420ac5e7a6eb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:37.455527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aadada-2047-4023-8d1c-420ac5e7a6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:37.456590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aadada-2047-4023-8d1c-420ac5e7a6eb None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:37.457958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aadada-2047-4023-8d1c-420ac5e7a6eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:37.458484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aadada-2047-4023-8d1c-420ac5e7a6eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:37.461096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aadada-2047-4023-8d1c-420ac5e7a6eb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:37.479057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 291/600] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:37.495179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aadada-2047-4023-8d1c-420ac5e7a6eb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:37.576741 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:37.626829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aadada-2047-4023-8d1c-420ac5e7a6eb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:37.666970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddcbd0-8641-4272-b2f9-fc5a14c7e997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:37.667929 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddcbd0-8641-4272-b2f9-fc5a14c7e997 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:37.668711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddcbd0-8641-4272-b2f9-fc5a14c7e997 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:37.669193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddcbd0-8641-4272-b2f9-fc5a14c7e997 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:37.677194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddcbd0-8641-4272-b2f9-fc5a14c7e997 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:37.678940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddcbd0-8641-4272-b2f9-fc5a14c7e997 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e2ddcbd0-8641-4272-b2f9-fc5a14c7e997 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:37.693197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:37.694117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:37.695022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:37.695022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:37.695022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:37.695022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:37.695022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:37.695022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:37.695022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:37.695022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:37.698266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 292/601] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:37.701033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca938413-01cc-4c3b-b277-5ebcdc238214 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:37.703707 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddcbd0-8641-4272-b2f9-fc5a14c7e997 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:37.766483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:37.823178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddcbd0-8641-4272-b2f9-fc5a14c7e997 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:37.838582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb3426-d3de-4521-a8c4-6cd2a4a04189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:37.838582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb3426-d3de-4521-a8c4-6cd2a4a04189 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:37.838848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb3426-d3de-4521-a8c4-6cd2a4a04189 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:37.838920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb3426-d3de-4521-a8c4-6cd2a4a04189 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:37.839612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb3426-d3de-4521-a8c4-6cd2a4a04189 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:37.839921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb3426-d3de-4521-a8c4-6cd2a4a04189 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:37.841300 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb3426-d3de-4521-a8c4-6cd2a4a04189 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:37.866678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe01db6-f7e5-48fb-8a58-15cf4f1dd6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:37.870570 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe01db6-f7e5-48fb-8a58-15cf4f1dd6fb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:37.871832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe01db6-f7e5-48fb-8a58-15cf4f1dd6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:37.872376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe01db6-f7e5-48fb-8a58-15cf4f1dd6fb None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:37.874502 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe01db6-f7e5-48fb-8a58-15cf4f1dd6fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:37.875026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe01db6-f7e5-48fb-8a58-15cf4f1dd6fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:37.875959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 310/602] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:37.876965 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe01db6-f7e5-48fb-8a58-15cf4f1dd6fb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:37.877267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89985d5-a22d-469f-bb71-205f4733e270 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:37.877557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89985d5-a22d-469f-bb71-205f4733e270 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:37.877838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89985d5-a22d-469f-bb71-205f4733e270 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:37.883621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 311/603] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:37.885441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89985d5-a22d-469f-bb71-205f4733e270 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:37.894236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 293/604] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:37.895807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52926e2-019d-4eea-ab4f-4f2ba239c60c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:37.896338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52926e2-019d-4eea-ab4f-4f2ba239c60c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:37.896839 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52926e2-019d-4eea-ab4f-4f2ba239c60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:37.921482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b52926e2-019d-4eea-ab4f-4f2ba239c60c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:37.923553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b52926e2-019d-4eea-ab4f-4f2ba239c60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:38.020557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 294/605] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:38.021683 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52926e2-019d-4eea-ab4f-4f2ba239c60c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:38.070580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:38.133999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52926e2-019d-4eea-ab4f-4f2ba239c60c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:38.163223 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89985d5-a22d-469f-bb71-205f4733e270 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:38.177763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8622142-1f59-42a9-a9ab-f291351114e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:38.178876 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8622142-1f59-42a9-a9ab-f291351114e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:38.179437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8622142-1f59-42a9-a9ab-f291351114e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:38.179983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8622142-1f59-42a9-a9ab-f291351114e0 None admin] PATH_INFO: `/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/495c489f00d74b53ac02c2cbbb323c38/roles/74131e9234dc48a2b55332d15e8c6462` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:38.195764 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:38.198268 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8622142-1f59-42a9-a9ab-f291351114e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ddf7d6f2424a85a741d017497b49c9, user_id=495c489f00d74b53ac02c2cbbb323c38, role_id=74131e9234dc48a2b55332d15e8c6462)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:38.200853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8622142-1f59-42a9-a9ab-f291351114e0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:38.201425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac19d7-3812-4bee-862e-c2b86e654e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:38.202077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac19d7-3812-4bee-862e-c2b86e654e32 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:38.202361 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac19d7-3812-4bee-862e-c2b86e654e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:38.202638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac19d7-3812-4bee-862e-c2b86e654e32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:38.211269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac19d7-3812-4bee-862e-c2b86e654e32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:38.213605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac19d7-3812-4bee-862e-c2b86e654e32 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:38.221909 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 295/606] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:24:38 2022] PUT /identity/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/495c489f00d74b53ac02c2cbbb323c38/roles/74131e9234dc48a2b55332d15e8c6462 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:24:38.223194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8622142-1f59-42a9-a9ab-f291351114e0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-16ac19d7-3812-4bee-862e-c2b86e654e32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:38.226907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:38.227993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:38.228882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:38.228882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:38.228882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:38.228882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:38.228882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:38.228882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:38.228882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:38.228882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:38.228882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 312/607] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:38.236953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac19d7-3812-4bee-862e-c2b86e654e32 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:38.374039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac19d7-3812-4bee-862e-c2b86e654e32 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:38.380664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:38.411095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8622142-1f59-42a9-a9ab-f291351114e0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:38.418414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5a12a-7e24-4d7d-b788-888624bfe6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:38.419746 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5a12a-7e24-4d7d-b788-888624bfe6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:38.420245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5a12a-7e24-4d7d-b788-888624bfe6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:38.421117 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5a12a-7e24-4d7d-b788-888624bfe6d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:38.422104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5a12a-7e24-4d7d-b788-888624bfe6d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:38.422906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5a12a-7e24-4d7d-b788-888624bfe6d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:38.425039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5a12a-7e24-4d7d-b788-888624bfe6d6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:38.441919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 313/608] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:38.447998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5a12a-7e24-4d7d-b788-888624bfe6d6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:38.453810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9b633-41fa-45a1-81b1-2edd28c78e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:38.454460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9b633-41fa-45a1-81b1-2edd28c78e01 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:38.454744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9b633-41fa-45a1-81b1-2edd28c78e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:38.455021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9b633-41fa-45a1-81b1-2edd28c78e01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:38.468773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9b633-41fa-45a1-81b1-2edd28c78e01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3bc9b633-41fa-45a1-81b1-2edd28c78e01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:38.474614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:38.475614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:38.476528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 296/609] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:38.481911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9b633-41fa-45a1-81b1-2edd28c78e01 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:38.525757 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:38.615380 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5a12a-7e24-4d7d-b788-888624bfe6d6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:38.653349 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9b633-41fa-45a1-81b1-2edd28c78e01 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:38.708325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4703f5a-b73a-42cf-905d-7fdb7eb56b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:38.709371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4703f5a-b73a-42cf-905d-7fdb7eb56b01 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:38.709860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4703f5a-b73a-42cf-905d-7fdb7eb56b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:38.710341 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4703f5a-b73a-42cf-905d-7fdb7eb56b01 None admin] PATH_INFO: `/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/138ede2854894a3b995435980965911c/roles/74131e9234dc48a2b55332d15e8c6462` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:38.710686 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5609a26-e5d1-4054-9427-68e1bd096d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:38.711346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5609a26-e5d1-4054-9427-68e1bd096d5b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:38.711711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5609a26-e5d1-4054-9427-68e1bd096d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:38.711991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5609a26-e5d1-4054-9427-68e1bd096d5b None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:38.719087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5609a26-e5d1-4054-9427-68e1bd096d5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:38.719087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5609a26-e5d1-4054-9427-68e1bd096d5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:38.719087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5609a26-e5d1-4054-9427-68e1bd096d5b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:38.724060 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 297/610] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:38.730078 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5609a26-e5d1-4054-9427-68e1bd096d5b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:38.745947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4703f5a-b73a-42cf-905d-7fdb7eb56b01 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ddf7d6f2424a85a741d017497b49c9, user_id=138ede2854894a3b995435980965911c, role_id=74131e9234dc48a2b55332d15e8c6462)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:38.747422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4703f5a-b73a-42cf-905d-7fdb7eb56b01 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:38.756620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:38.766567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 314/611] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:24:38 2022] PUT /identity/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/138ede2854894a3b995435980965911c/roles/74131e9234dc48a2b55332d15e8c6462 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:24:38.882870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5609a26-e5d1-4054-9427-68e1bd096d5b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:38.923556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79135a-e3fe-4175-acd4-5eb6fc37d658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:38.924212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79135a-e3fe-4175-acd4-5eb6fc37d658 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:38.924494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79135a-e3fe-4175-acd4-5eb6fc37d658 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:38.924767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79135a-e3fe-4175-acd4-5eb6fc37d658 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:38.938058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79135a-e3fe-4175-acd4-5eb6fc37d658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:38.939754 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79135a-e3fe-4175-acd4-5eb6fc37d658 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5e79135a-e3fe-4175-acd4-5eb6fc37d658 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:38.972697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:38.973654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:38.974536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:38.974536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:38.974536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:38.974536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:38.974536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 298/612] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:38.980014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4703f5a-b73a-42cf-905d-7fdb7eb56b01 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:39.107115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4703f5a-b73a-42cf-905d-7fdb7eb56b01 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:39.109095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:39.145618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c466ef4-3bf1-4020-af4f-c6efba12aa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:39.146242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c466ef4-3bf1-4020-af4f-c6efba12aa26 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:39.146529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c466ef4-3bf1-4020-af4f-c6efba12aa26 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:39.146807 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c466ef4-3bf1-4020-af4f-c6efba12aa26 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:39.147547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c466ef4-3bf1-4020-af4f-c6efba12aa26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:39.147859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c466ef4-3bf1-4020-af4f-c6efba12aa26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:39.149229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c466ef4-3bf1-4020-af4f-c6efba12aa26 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:39.162123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 315/613] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:39.167973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79135a-e3fe-4175-acd4-5eb6fc37d658 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:39.284532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:39.336856 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533f388-2e64-4164-a68a-8905fab70d57 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:39.338121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533f388-2e64-4164-a68a-8905fab70d57 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:39.338722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533f388-2e64-4164-a68a-8905fab70d57 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:39.341908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 316/614] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:39.346401 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2779b-4979-4beb-a9cf-f2e349d6b436 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:39.346625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2779b-4979-4beb-a9cf-f2e349d6b436 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:39.346845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2779b-4979-4beb-a9cf-f2e349d6b436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:39.355343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79135a-e3fe-4175-acd4-5eb6fc37d658 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:39.388951 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-07f2779b-4979-4beb-a9cf-f2e349d6b436 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:39.390256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-07f2779b-4979-4beb-a9cf-f2e349d6b436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:39.420171 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152c24a-54d3-4eec-9c35-1c4a6a4f7ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:39.421052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152c24a-54d3-4eec-9c35-1c4a6a4f7ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:39.421442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152c24a-54d3-4eec-9c35-1c4a6a4f7ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:39.421828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152c24a-54d3-4eec-9c35-1c4a6a4f7ce8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:39.433499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152c24a-54d3-4eec-9c35-1c4a6a4f7ce8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:39.435710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152c24a-54d3-4eec-9c35-1c4a6a4f7ce8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b152c24a-54d3-4eec-9c35-1c4a6a4f7ce8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:39.454451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:39.455496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:39.456471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:39.456471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:39.456471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:39.456471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:39.456471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:39.456471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:39.456471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:39.456471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:39.456471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:24:39.456471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:39.456994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 299/615] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:39.463693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152c24a-54d3-4eec-9c35-1c4a6a4f7ce8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:39.586582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152c24a-54d3-4eec-9c35-1c4a6a4f7ce8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:39.594093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 317/616] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:39.595519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14eb55-64fa-413b-82a3-d0cbaf14bb4a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:39.595814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14eb55-64fa-413b-82a3-d0cbaf14bb4a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:39.596094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14eb55-64fa-413b-82a3-d0cbaf14bb4a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:39.603501 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 318/617] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:39.609305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab58f2-2804-422b-9780-a20a4ae58dc2 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:39.609606 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab58f2-2804-422b-9780-a20a4ae58dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:39.609889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab58f2-2804-422b-9780-a20a4ae58dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:39.616691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:39.633481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc45a1d-c48d-48ce-b9a6-a12e40d519ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:39.635348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc45a1d-c48d-48ce-b9a6-a12e40d519ee None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:39.635764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc45a1d-c48d-48ce-b9a6-a12e40d519ee None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:39.636546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc45a1d-c48d-48ce-b9a6-a12e40d519ee None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:39.637646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc45a1d-c48d-48ce-b9a6-a12e40d519ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:39.637980 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc45a1d-c48d-48ce-b9a6-a12e40d519ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:39.640183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc45a1d-c48d-48ce-b9a6-a12e40d519ee None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:39.641756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d3ab58f2-2804-422b-9780-a20a4ae58dc2 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:39.642943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d3ab58f2-2804-422b-9780-a20a4ae58dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:39.665207 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 300/618] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Aug 1 07:24:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:39.666756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc45a1d-c48d-48ce-b9a6-a12e40d519ee None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:39.764142 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 319/619] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:39.765124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab58f2-2804-422b-9780-a20a4ae58dc2 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:39.797134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:39.806843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc45a1d-c48d-48ce-b9a6-a12e40d519ee None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:39.851801 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae73c-0078-4cab-923a-c322c1f29409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:39.852680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae73c-0078-4cab-923a-c322c1f29409 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:39.854287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae73c-0078-4cab-923a-c322c1f29409 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:39.854728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae73c-0078-4cab-923a-c322c1f29409 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:39.867996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eae73c-0078-4cab-923a-c322c1f29409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-08eae73c-0078-4cab-923a-c322c1f29409 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:39.874733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:39.875854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:39.876863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:39.876863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:39.876863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:39.876863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:39.876863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:39.876863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:39.876863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:39.876863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:39.876863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:39.876863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:39.876863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:39.876863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:39.879585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 301/620] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:39.881059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae73c-0078-4cab-923a-c322c1f29409 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:39.911452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab58f2-2804-422b-9780-a20a4ae58dc2 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:39.943231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:39.965044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f86f6-c897-4405-8124-a02e1f7884c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:39.967096 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f86f6-c897-4405-8124-a02e1f7884c7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:39.967538 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f86f6-c897-4405-8124-a02e1f7884c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:39.967990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f86f6-c897-4405-8124-a02e1f7884c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:39.983184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f86f6-c897-4405-8124-a02e1f7884c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:39.985677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f86f6-c897-4405-8124-a02e1f7884c7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-952f86f6-c897-4405-8124-a02e1f7884c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:40.008124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:40.010256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:40.011355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:40.011355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:40.011355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:40.011355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:40.013972 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 320/621] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:40.015091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f86f6-c897-4405-8124-a02e1f7884c7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:40.027713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae73c-0078-4cab-923a-c322c1f29409 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:40.071266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ff56d-fb1b-4e02-8630-fa10d3be68fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:40.072434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ff56d-fb1b-4e02-8630-fa10d3be68fe None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:40.072924 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ff56d-fb1b-4e02-8630-fa10d3be68fe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:40.073363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ff56d-fb1b-4e02-8630-fa10d3be68fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:40.090342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ff56d-fb1b-4e02-8630-fa10d3be68fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a84ff56d-fb1b-4e02-8630-fa10d3be68fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:40.098865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:40.100004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:40.101009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:40.101009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:40.101009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:40.101009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:40.101009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:40.101009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:40.101009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:40.101009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:40.101009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:40.101009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:40.101009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:40.101009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:40.104423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 302/622] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:40.105104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ff56d-fb1b-4e02-8630-fa10d3be68fe None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:40.119355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:40.156038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f86f6-c897-4405-8124-a02e1f7884c7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:40.210953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e3999-76e3-45cb-bb17-1ee0fc964f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:40.211960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e3999-76e3-45cb-bb17-1ee0fc964f77 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:40.212649 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e3999-76e3-45cb-bb17-1ee0fc964f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:40.213202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e3999-76e3-45cb-bb17-1ee0fc964f77 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:40.214130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e3999-76e3-45cb-bb17-1ee0fc964f77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:40.214688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e3999-76e3-45cb-bb17-1ee0fc964f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:40.216946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e3999-76e3-45cb-bb17-1ee0fc964f77 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:40.230685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 321/623] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:40.231849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e3999-76e3-45cb-bb17-1ee0fc964f77 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:40.249252 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ff56d-fb1b-4e02-8630-fa10d3be68fe None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:40.251446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:40.295738 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ed956-b941-497c-9344-06e4eb86ef1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:40.296421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ed956-b941-497c-9344-06e4eb86ef1c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:40.296715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ed956-b941-497c-9344-06e4eb86ef1c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:40.297040 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ed956-b941-497c-9344-06e4eb86ef1c None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:40.297960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ed956-b941-497c-9344-06e4eb86ef1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:40.298096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ed956-b941-497c-9344-06e4eb86ef1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:40.299466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ed956-b941-497c-9344-06e4eb86ef1c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:40.316759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 303/624] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:40.317688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ed956-b941-497c-9344-06e4eb86ef1c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:40.363743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e3999-76e3-45cb-bb17-1ee0fc964f77 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:40.404967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd684e44-7712-47b1-b247-7ecd5573cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:40.405985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd684e44-7712-47b1-b247-7ecd5573cb70 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:40.406456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd684e44-7712-47b1-b247-7ecd5573cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:40.406906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd684e44-7712-47b1-b247-7ecd5573cb70 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:40.407866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd684e44-7712-47b1-b247-7ecd5573cb70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:40.408850 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd684e44-7712-47b1-b247-7ecd5573cb70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:40.410557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd684e44-7712-47b1-b247-7ecd5573cb70 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:40.413610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:40.423442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 322/625] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:40.424453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ed956-b941-497c-9344-06e4eb86ef1c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:40.425080 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd684e44-7712-47b1-b247-7ecd5573cb70 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:40.460591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dc702-5ddb-4278-90cf-0aa7b90ca323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:40.461241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dc702-5ddb-4278-90cf-0aa7b90ca323 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:40.461526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dc702-5ddb-4278-90cf-0aa7b90ca323 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:40.461799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dc702-5ddb-4278-90cf-0aa7b90ca323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:40.473420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627dc702-5ddb-4278-90cf-0aa7b90ca323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:40.474947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627dc702-5ddb-4278-90cf-0aa7b90ca323 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-627dc702-5ddb-4278-90cf-0aa7b90ca323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:40.490728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:40.491977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:40.493035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:40.493035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:40.493035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:40.493035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:40.496847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 304/626] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:40.497724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dc702-5ddb-4278-90cf-0aa7b90ca323 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:40.514878 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:40.551439 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd684e44-7712-47b1-b247-7ecd5573cb70 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:40.592123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e3231-7a7b-419c-8e1f-a83cfecc9418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:40.593048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e3231-7a7b-419c-8e1f-a83cfecc9418 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:40.593544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e3231-7a7b-419c-8e1f-a83cfecc9418 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:40.593988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e3231-7a7b-419c-8e1f-a83cfecc9418 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:40.602295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e3231-7a7b-419c-8e1f-a83cfecc9418 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:40.604266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dc702-5ddb-4278-90cf-0aa7b90ca323 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:40.604569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e3231-7a7b-419c-8e1f-a83cfecc9418 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-d61e3231-7a7b-419c-8e1f-a83cfecc9418 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:40.619444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:40.620781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:40.621895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:40.621895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:40.621895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:40.621895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:40.621895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:40.621895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:40.621895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:40.621895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:40.624539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 323/627] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:40.625522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e3231-7a7b-419c-8e1f-a83cfecc9418 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:40.652811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26306d-cdc6-4e63-87db-ee51fa6de788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:40.653461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26306d-cdc6-4e63-87db-ee51fa6de788 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:40.653747 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26306d-cdc6-4e63-87db-ee51fa6de788 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:40.654026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26306d-cdc6-4e63-87db-ee51fa6de788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:40.664750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26306d-cdc6-4e63-87db-ee51fa6de788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:40.666303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26306d-cdc6-4e63-87db-ee51fa6de788 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-af26306d-cdc6-4e63-87db-ee51fa6de788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:40.680555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:40.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:40.682941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:40.682941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:40.682941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:40.682941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:40.682941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 305/628] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:40.683262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26306d-cdc6-4e63-87db-ee51fa6de788 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:40.704197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:40.750852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:40.751727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e3231-7a7b-419c-8e1f-a83cfecc9418 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:40.788584 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b986c-cb80-4e12-b7e1-3f127d5d8cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:40.788924 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26306d-cdc6-4e63-87db-ee51fa6de788 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:40.789257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b986c-cb80-4e12-b7e1-3f127d5d8cba None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:40.789568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b986c-cb80-4e12-b7e1-3f127d5d8cba None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:40.789869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b986c-cb80-4e12-b7e1-3f127d5d8cba None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:40.790648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b986c-cb80-4e12-b7e1-3f127d5d8cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:40.790973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b986c-cb80-4e12-b7e1-3f127d5d8cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:40.792470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b986c-cb80-4e12-b7e1-3f127d5d8cba None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:40.810041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 324/629] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:40.810041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b9781-0700-452c-84b0-e62ca5ad5d1b None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:40.810041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b9781-0700-452c-84b0-e62ca5ad5d1b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:40.810041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b9781-0700-452c-84b0-e62ca5ad5d1b None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:40.810041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 325/630] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:40.810041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b9781-0700-452c-84b0-e62ca5ad5d1b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:40.832985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef09ee5-735c-4185-8be3-58d5ea1a4d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:40.833620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef09ee5-735c-4185-8be3-58d5ea1a4d3a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:40.833908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef09ee5-735c-4185-8be3-58d5ea1a4d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:40.834190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef09ee5-735c-4185-8be3-58d5ea1a4d3a None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:40.834944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef09ee5-735c-4185-8be3-58d5ea1a4d3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:40.835288 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef09ee5-735c-4185-8be3-58d5ea1a4d3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:40.836732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef09ee5-735c-4185-8be3-58d5ea1a4d3a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:40.851172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 306/631] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:40.857656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef09ee5-735c-4185-8be3-58d5ea1a4d3a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:40.925519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b9781-0700-452c-84b0-e62ca5ad5d1b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:40.944833 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:40.967335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef09ee5-735c-4185-8be3-58d5ea1a4d3a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:40.971399 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca152c5-0630-47f5-ad98-61ba493e33eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:40.972103 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca152c5-0630-47f5-ad98-61ba493e33eb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:40.972457 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca152c5-0630-47f5-ad98-61ba493e33eb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:40.972802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca152c5-0630-47f5-ad98-61ba493e33eb None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:40.973564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca152c5-0630-47f5-ad98-61ba493e33eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:40.973963 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca152c5-0630-47f5-ad98-61ba493e33eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:40.975436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca152c5-0630-47f5-ad98-61ba493e33eb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:40.993154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 326/632] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:40.993154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33288e6b-5d22-4567-9dd7-d18c591bddbf None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:40.993154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33288e6b-5d22-4567-9dd7-d18c591bddbf None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:40.993154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33288e6b-5d22-4567-9dd7-d18c591bddbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:41.002101 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:41.010860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-33288e6b-5d22-4567-9dd7-d18c591bddbf None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:41.012084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-33288e6b-5d22-4567-9dd7-d18c591bddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:41.013875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b1f86-9ff9-4c35-972a-3e425e1aa950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:41.014565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b1f86-9ff9-4c35-972a-3e425e1aa950 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:41.014876 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b1f86-9ff9-4c35-972a-3e425e1aa950 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:41.015840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b1f86-9ff9-4c35-972a-3e425e1aa950 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:41.026175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b1f86-9ff9-4c35-972a-3e425e1aa950 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:41.027918 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b1f86-9ff9-4c35-972a-3e425e1aa950 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fb1b1f86-9ff9-4c35-972a-3e425e1aa950 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:41.041420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:41.042445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:41.044128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:41.044128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:41.044128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:41.044128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:41.044128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:41.044128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:41.044128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:41.044128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:41.044128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:24:41.044128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:41.044128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 307/633] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Aug 1 07:24:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:41.044128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b1f86-9ff9-4c35-972a-3e425e1aa950 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:41.094850 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 327/634] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:41.095721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33288e6b-5d22-4567-9dd7-d18c591bddbf None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:41.169641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b1f86-9ff9-4c35-972a-3e425e1aa950 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:41.206016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8b03a-de06-48c9-9c8b-cdc0b8d5afb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:41.206733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8b03a-de06-48c9-9c8b-cdc0b8d5afb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:41.207104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8b03a-de06-48c9-9c8b-cdc0b8d5afb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:41.207298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33288e6b-5d22-4567-9dd7-d18c591bddbf None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:41.207436 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8b03a-de06-48c9-9c8b-cdc0b8d5afb4 None admin] PATH_INFO: `/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/2ba2655c2eef408498c0298d432263f6/roles/74131e9234dc48a2b55332d15e8c6462` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:41.219270 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:41.224303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8b03a-de06-48c9-9c8b-cdc0b8d5afb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ddf7d6f2424a85a741d017497b49c9, user_id=2ba2655c2eef408498c0298d432263f6, role_id=74131e9234dc48a2b55332d15e8c6462)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:41.225762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8b03a-de06-48c9-9c8b-cdc0b8d5afb4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:41.238835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 308/635] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:24:41 2022] PUT /identity/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/2ba2655c2eef408498c0298d432263f6/roles/74131e9234dc48a2b55332d15e8c6462 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:24:41.239685 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8b03a-de06-48c9-9c8b-cdc0b8d5afb4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:41.256994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833edf3-5df6-4c5c-b694-7472df85629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:41.258097 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833edf3-5df6-4c5c-b694-7472df85629d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:41.258596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833edf3-5df6-4c5c-b694-7472df85629d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:41.259643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833edf3-5df6-4c5c-b694-7472df85629d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:41.268863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833edf3-5df6-4c5c-b694-7472df85629d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:41.270951 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833edf3-5df6-4c5c-b694-7472df85629d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-6833edf3-5df6-4c5c-b694-7472df85629d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:41.284542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:41.285625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:41.286569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:41.286569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:41.286569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:41.286569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:41.286569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:41.286569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:41.286569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:41.286569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:41.286569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 07:24:41.286569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:41.290555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 328/636] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:41.291539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833edf3-5df6-4c5c-b694-7472df85629d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:41.295452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:41.397995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8b03a-de06-48c9-9c8b-cdc0b8d5afb4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:41.415529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833edf3-5df6-4c5c-b694-7472df85629d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:41.444384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bda173-087d-4abd-a140-5040c1a55e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:41.445854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bda173-087d-4abd-a140-5040c1a55e9b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:41.446318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bda173-087d-4abd-a140-5040c1a55e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:41.447022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bda173-087d-4abd-a140-5040c1a55e9b None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:41.448649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bda173-087d-4abd-a140-5040c1a55e9b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:41.449119 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bda173-087d-4abd-a140-5040c1a55e9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:41.450609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bda173-087d-4abd-a140-5040c1a55e9b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:41.459241 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47848de2-ae49-4821-8adb-3286a33d6aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:41.459815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47848de2-ae49-4821-8adb-3286a33d6aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:41.460102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47848de2-ae49-4821-8adb-3286a33d6aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:41.460370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47848de2-ae49-4821-8adb-3286a33d6aa2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:41.469773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 309/637] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Aug 1 07:24:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:41.471176 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bda173-087d-4abd-a140-5040c1a55e9b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:41.476184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47848de2-ae49-4821-8adb-3286a33d6aa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-47848de2-ae49-4821-8adb-3286a33d6aa2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:41.481627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:41.482920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:41.484059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 329/638] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:41.484941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54bf42-6af3-421c-8051-a3424f4780ac None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:41.484941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54bf42-6af3-421c-8051-a3424f4780ac None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:41.484941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54bf42-6af3-421c-8051-a3424f4780ac None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:41.486175 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 330/639] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:41.491097 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54bf42-6af3-421c-8051-a3424f4780ac None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:41.535024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:41.569994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:41.579855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bda173-087d-4abd-a140-5040c1a55e9b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:41.615235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dee244-c919-4617-a74c-583837355617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:41.615859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dee244-c919-4617-a74c-583837355617 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:41.616144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dee244-c919-4617-a74c-583837355617 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:41.616420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dee244-c919-4617-a74c-583837355617 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:41.617234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dee244-c919-4617-a74c-583837355617 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:41.617608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dee244-c919-4617-a74c-583837355617 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:41.620123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dee244-c919-4617-a74c-583837355617 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:41.638691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 310/640] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Aug 1 07:24:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:41.639574 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dee244-c919-4617-a74c-583837355617 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:41.651792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54bf42-6af3-421c-8051-a3424f4780ac None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:41.688577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062eda41-f4e3-4673-8246-6045b3851bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:41.689212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062eda41-f4e3-4673-8246-6045b3851bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:41.689493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062eda41-f4e3-4673-8246-6045b3851bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:41.689770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062eda41-f4e3-4673-8246-6045b3851bb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:41.756304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dee244-c919-4617-a74c-583837355617 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:41.774733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062eda41-f4e3-4673-8246-6045b3851bb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:41.776559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062eda41-f4e3-4673-8246-6045b3851bb3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:41.794689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cba69a-a132-48f9-a25c-41473a7014b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:41.795491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cba69a-a132-48f9-a25c-41473a7014b2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:41.795848 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cba69a-a132-48f9-a25c-41473a7014b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:41.796155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cba69a-a132-48f9-a25c-41473a7014b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:41.797029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cba69a-a132-48f9-a25c-41473a7014b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:41.797443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cba69a-a132-48f9-a25c-41473a7014b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:41.799024 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cba69a-a132-48f9-a25c-41473a7014b2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:41.808333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:41.810432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 311/641] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:41.811981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69581568-2562-4dff-a067-aa76e449e26b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:41.812326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69581568-2562-4dff-a067-aa76e449e26b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:41.812664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69581568-2562-4dff-a067-aa76e449e26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:41.839269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-69581568-2562-4dff-a067-aa76e449e26b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:41.841192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-69581568-2562-4dff-a067-aa76e449e26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:41.856632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:41.903614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 331/642] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:41.903614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062eda41-f4e3-4673-8246-6045b3851bb3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:41.923122 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 312/643] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:41.923122 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69581568-2562-4dff-a067-aa76e449e26b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:42.039123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69581568-2562-4dff-a067-aa76e449e26b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:42.062782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062eda41-f4e3-4673-8246-6045b3851bb3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:42.075241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca211c3-46ca-4696-b854-a28890f4b466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:42.075895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca211c3-46ca-4696-b854-a28890f4b466 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:42.076183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca211c3-46ca-4696-b854-a28890f4b466 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:42.076460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca211c3-46ca-4696-b854-a28890f4b466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:42.086718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca211c3-46ca-4696-b854-a28890f4b466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:42.088391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca211c3-46ca-4696-b854-a28890f4b466 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:42.091125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3ca211c3-46ca-4696-b854-a28890f4b466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:42.104996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:42.106086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:42.107053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:42.107053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:42.107053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:42.107053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:42.107053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 313/644] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:42.107053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca211c3-46ca-4696-b854-a28890f4b466 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:42.114492 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:42.125342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf967d-ad70-4a1f-b6c1-5665a461a4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:42.126636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf967d-ad70-4a1f-b6c1-5665a461a4fd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:42.127095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf967d-ad70-4a1f-b6c1-5665a461a4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:42.127542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf967d-ad70-4a1f-b6c1-5665a461a4fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:42.203449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf967d-ad70-4a1f-b6c1-5665a461a4fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:42.205399 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf967d-ad70-4a1f-b6c1-5665a461a4fd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:42.245724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca211c3-46ca-4696-b854-a28890f4b466 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:42.293051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06026fc1-3390-48b6-99cb-ba7b44b3a727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:42.293829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06026fc1-3390-48b6-99cb-ba7b44b3a727 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:42.294239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06026fc1-3390-48b6-99cb-ba7b44b3a727 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:42.294735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06026fc1-3390-48b6-99cb-ba7b44b3a727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:42.313704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06026fc1-3390-48b6-99cb-ba7b44b3a727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:42.315780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06026fc1-3390-48b6-99cb-ba7b44b3a727 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:42.318499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 332/645] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:42.319302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf967d-ad70-4a1f-b6c1-5665a461a4fd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-06026fc1-3390-48b6-99cb-ba7b44b3a727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:42.336715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:42.338366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:42.339443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:42.339443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:42.339443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:42.339443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:42.339443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 314/646] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:42.340283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06026fc1-3390-48b6-99cb-ba7b44b3a727 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:42.358646 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:42.406036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:42.454923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf967d-ad70-4a1f-b6c1-5665a461a4fd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:42.479676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06026fc1-3390-48b6-99cb-ba7b44b3a727 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:42.493232 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07581167-3725-4f88-95ba-001edba973a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:42.493700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07581167-3725-4f88-95ba-001edba973a2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:42.493969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07581167-3725-4f88-95ba-001edba973a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:42.494245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07581167-3725-4f88-95ba-001edba973a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:42.504580 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07581167-3725-4f88-95ba-001edba973a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-07581167-3725-4f88-95ba-001edba973a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:42.510348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:42.511337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 333/647] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:42.512180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07581167-3725-4f88-95ba-001edba973a2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:42.530137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2e4fd-4455-4694-aaed-69596c05de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:42.530828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2e4fd-4455-4694-aaed-69596c05de4e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:42.531237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2e4fd-4455-4694-aaed-69596c05de4e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:42.531579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2e4fd-4455-4694-aaed-69596c05de4e None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:42.532401 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2e4fd-4455-4694-aaed-69596c05de4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:42.532818 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2e4fd-4455-4694-aaed-69596c05de4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:42.534449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2e4fd-4455-4694-aaed-69596c05de4e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:42.554812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 315/648] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:42.554812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2e4fd-4455-4694-aaed-69596c05de4e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:42.608140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:42.622072 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07581167-3725-4f88-95ba-001edba973a2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:42.658559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73942bcc-91ac-4d10-bdbb-257b0079786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:42.659662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73942bcc-91ac-4d10-bdbb-257b0079786b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:42.660167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73942bcc-91ac-4d10-bdbb-257b0079786b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:42.660673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73942bcc-91ac-4d10-bdbb-257b0079786b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:42.670732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2e4fd-4455-4694-aaed-69596c05de4e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:42.677851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73942bcc-91ac-4d10-bdbb-257b0079786b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:42.679457 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73942bcc-91ac-4d10-bdbb-257b0079786b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:42.681872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-73942bcc-91ac-4d10-bdbb-257b0079786b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:42.696647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:42.697852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:42.698737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:42.698737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:42.698737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:42.698737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:42.698737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 334/649] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:42.699102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73942bcc-91ac-4d10-bdbb-257b0079786b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:42.714358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93684f-be5e-4798-bb02-438e6eb78ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:42.714981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93684f-be5e-4798-bb02-438e6eb78ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:42.715308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93684f-be5e-4798-bb02-438e6eb78ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:42.715582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93684f-be5e-4798-bb02-438e6eb78ef7 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:42.716279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93684f-be5e-4798-bb02-438e6eb78ef7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:42.716571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93684f-be5e-4798-bb02-438e6eb78ef7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:42.717930 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93684f-be5e-4798-bb02-438e6eb78ef7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:42.734556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 316/650] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:42.734556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93684f-be5e-4798-bb02-438e6eb78ef7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:42.818823 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73942bcc-91ac-4d10-bdbb-257b0079786b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:42.848163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93684f-be5e-4798-bb02-438e6eb78ef7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:42.861309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37948cad-99d1-453b-8f16-5a2a1bd554af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:42.862155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37948cad-99d1-453b-8f16-5a2a1bd554af None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:42.862437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37948cad-99d1-453b-8f16-5a2a1bd554af None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:42.862736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37948cad-99d1-453b-8f16-5a2a1bd554af None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:42.863488 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37948cad-99d1-453b-8f16-5a2a1bd554af None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:42.863798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37948cad-99d1-453b-8f16-5a2a1bd554af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:42.865222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37948cad-99d1-453b-8f16-5a2a1bd554af None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:42.879781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 335/651] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:42.879781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37948cad-99d1-453b-8f16-5a2a1bd554af None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:42.879781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:42.894165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf600f6-bb29-4c40-9d6d-7c2c381590a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:42.894898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf600f6-bb29-4c40-9d6d-7c2c381590a3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:42.895277 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf600f6-bb29-4c40-9d6d-7c2c381590a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:42.895626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf600f6-bb29-4c40-9d6d-7c2c381590a3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:42.904461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf600f6-bb29-4c40-9d6d-7c2c381590a3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:42.905891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf600f6-bb29-4c40-9d6d-7c2c381590a3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9bf600f6-bb29-4c40-9d6d-7c2c381590a3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:42.917665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:42.918691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:42.919653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:42.919653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:42.919653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:42.919653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:42.919653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:42.919653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:42.919653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:42.919653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:42.919653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:24:42.919653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:42.919653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 317/652] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:42.919653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf600f6-bb29-4c40-9d6d-7c2c381590a3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:42.966453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:43.001521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37948cad-99d1-453b-8f16-5a2a1bd554af None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:43.038590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be82e9c-77ca-4fa9-bbf0-f308be9e8727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:43.039238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be82e9c-77ca-4fa9-bbf0-f308be9e8727 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.039522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be82e9c-77ca-4fa9-bbf0-f308be9e8727 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.039796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be82e9c-77ca-4fa9-bbf0-f308be9e8727 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.041198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be82e9c-77ca-4fa9-bbf0-f308be9e8727 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:43.041504 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be82e9c-77ca-4fa9-bbf0-f308be9e8727 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:43.042876 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be82e9c-77ca-4fa9-bbf0-f308be9e8727 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:43.045306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf600f6-bb29-4c40-9d6d-7c2c381590a3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:43.054996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 336/653] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:43.056888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be82e9c-77ca-4fa9-bbf0-f308be9e8727 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:43.088656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaae6e7-4af1-4e55-a7bb-d100f4ee890e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:43.089697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaae6e7-4af1-4e55-a7bb-d100f4ee890e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.090248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaae6e7-4af1-4e55-a7bb-d100f4ee890e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.090533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaae6e7-4af1-4e55-a7bb-d100f4ee890e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.098560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaae6e7-4af1-4e55-a7bb-d100f4ee890e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:43.100087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaae6e7-4af1-4e55-a7bb-d100f4ee890e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2eaae6e7-4af1-4e55-a7bb-d100f4ee890e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:43.112887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:43.113949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:43.114935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:43.114935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:43.114935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:43.114935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:43.114935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:43.114935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:43.114935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:43.114935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:43.114935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 318/654] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:43.114935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaae6e7-4af1-4e55-a7bb-d100f4ee890e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:43.126584 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:43.172718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be82e9c-77ca-4fa9-bbf0-f308be9e8727 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:43.209012 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:43.211388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9aadfa-b125-4845-af72-15688144d171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:43.212082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9aadfa-b125-4845-af72-15688144d171 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.212617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9aadfa-b125-4845-af72-15688144d171 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.212898 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9aadfa-b125-4845-af72-15688144d171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.222648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaae6e7-4af1-4e55-a7bb-d100f4ee890e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:43.225985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9aadfa-b125-4845-af72-15688144d171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:43.226769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9aadfa-b125-4845-af72-15688144d171 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ee9aadfa-b125-4845-af72-15688144d171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:43.242712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:43.244578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:43.245631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:43.245631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:43.245631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:43.245631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:43.245631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 337/655] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:43.245631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9aadfa-b125-4845-af72-15688144d171 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:43.267521 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece0d4-9087-4103-95a0-4e5536f294b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:43.267892 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece0d4-9087-4103-95a0-4e5536f294b7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.268177 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece0d4-9087-4103-95a0-4e5536f294b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.268454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece0d4-9087-4103-95a0-4e5536f294b7 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.269249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece0d4-9087-4103-95a0-4e5536f294b7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:43.269555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece0d4-9087-4103-95a0-4e5536f294b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:43.270975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece0d4-9087-4103-95a0-4e5536f294b7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:43.290583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 319/656] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Aug 1 07:24:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:43.292232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece0d4-9087-4103-95a0-4e5536f294b7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:43.360541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9aadfa-b125-4845-af72-15688144d171 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:43.374703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:43.396094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf11bc-c249-4a45-b2c3-ea6a1c489c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:43.396721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf11bc-c249-4a45-b2c3-ea6a1c489c00 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.396997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf11bc-c249-4a45-b2c3-ea6a1c489c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.397275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf11bc-c249-4a45-b2c3-ea6a1c489c00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.404059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf11bc-c249-4a45-b2c3-ea6a1c489c00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:43.405501 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf11bc-c249-4a45-b2c3-ea6a1c489c00 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:43.409364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece0d4-9087-4103-95a0-4e5536f294b7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-30bf11bc-c249-4a45-b2c3-ea6a1c489c00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:43.417188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:43.418442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:43.419592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:43.419592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:43.419592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:43.419592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:43.419592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:43.419592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:43.419592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:43.419592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:43.419592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 338/657] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:43.419592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf11bc-c249-4a45-b2c3-ea6a1c489c00 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:43.446035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd2722-96f3-48c4-8892-b27f1f952237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:43.446854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd2722-96f3-48c4-8892-b27f1f952237 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.446854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd2722-96f3-48c4-8892-b27f1f952237 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.447246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd2722-96f3-48c4-8892-b27f1f952237 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.447992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd2722-96f3-48c4-8892-b27f1f952237 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:43.448300 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd2722-96f3-48c4-8892-b27f1f952237 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:43.449729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd2722-96f3-48c4-8892-b27f1f952237 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:43.461048 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 320/658] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:43.461864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd2722-96f3-48c4-8892-b27f1f952237 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:43.472879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:43.521975 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf11bc-c249-4a45-b2c3-ea6a1c489c00 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:43.556274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff488c-c8d3-40c0-bc59-15033b3fbe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:43.556925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff488c-c8d3-40c0-bc59-15033b3fbe5e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.557208 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff488c-c8d3-40c0-bc59-15033b3fbe5e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.557483 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff488c-c8d3-40c0-bc59-15033b3fbe5e None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.558167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff488c-c8d3-40c0-bc59-15033b3fbe5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:43.558471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff488c-c8d3-40c0-bc59-15033b3fbe5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:43.559904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff488c-c8d3-40c0-bc59-15033b3fbe5e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:43.568353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd2722-96f3-48c4-8892-b27f1f952237 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:43.573092 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 339/659] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:43.574319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5620111-ef5b-4c2b-924f-7021228c6412 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.574610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5620111-ef5b-4c2b-924f-7021228c6412 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.574891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5620111-ef5b-4c2b-924f-7021228c6412 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.576169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 340/660] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:43.576855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5620111-ef5b-4c2b-924f-7021228c6412 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:43.607500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1790a1-8be2-4915-af04-4fb4d9c32791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:43.608387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1790a1-8be2-4915-af04-4fb4d9c32791 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.608673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1790a1-8be2-4915-af04-4fb4d9c32791 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.608946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1790a1-8be2-4915-af04-4fb4d9c32791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.612736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:43.681152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1790a1-8be2-4915-af04-4fb4d9c32791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:43.682669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5620111-ef5b-4c2b-924f-7021228c6412 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:43.682796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1790a1-8be2-4915-af04-4fb4d9c32791 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:43.691245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:43.717950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58001e4c-698a-4b20-a4ad-2172612fd3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:43.718801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58001e4c-698a-4b20-a4ad-2172612fd3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.719096 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58001e4c-698a-4b20-a4ad-2172612fd3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.719383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58001e4c-698a-4b20-a4ad-2172612fd3f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.720129 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58001e4c-698a-4b20-a4ad-2172612fd3f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:43.720458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58001e4c-698a-4b20-a4ad-2172612fd3f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:43.721869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58001e4c-698a-4b20-a4ad-2172612fd3f6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:43.733493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 341/661] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:43.734299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58001e4c-698a-4b20-a4ad-2172612fd3f6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:43.784929 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 321/662] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:43.785774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1790a1-8be2-4915-af04-4fb4d9c32791 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:43.836070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:43.844649 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58001e4c-698a-4b20-a4ad-2172612fd3f6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:43.879196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc679859-12d0-47f1-beb6-126e77e7fb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:43.879741 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc679859-12d0-47f1-beb6-126e77e7fb6e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.880023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc679859-12d0-47f1-beb6-126e77e7fb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.880304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc679859-12d0-47f1-beb6-126e77e7fb6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.881086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc679859-12d0-47f1-beb6-126e77e7fb6e None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=abdcd6d09fee45c696449641ea36eae3) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:43.881397 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc679859-12d0-47f1-beb6-126e77e7fb6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:43.882831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc679859-12d0-47f1-beb6-126e77e7fb6e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:43.885456 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1790a1-8be2-4915-af04-4fb4d9c32791 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:43.888995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 342/663] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:43 2022] GET /identity/v3/role_assignments?user.id=abdcd6d09fee45c696449641ea36eae3&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:43.890236 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96eb1a0-b51e-42e5-9517-491cd14f0de7 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.890531 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96eb1a0-b51e-42e5-9517-491cd14f0de7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.890809 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96eb1a0-b51e-42e5-9517-491cd14f0de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.893200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:43.911767 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a96eb1a0-b51e-42e5-9517-491cd14f0de7 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:43.913135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a96eb1a0-b51e-42e5-9517-491cd14f0de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:43.926672 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713ec6d-54f9-44fd-a49a-61dde57e69f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:43.927472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713ec6d-54f9-44fd-a49a-61dde57e69f3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:43.927827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713ec6d-54f9-44fd-a49a-61dde57e69f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:43.928181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713ec6d-54f9-44fd-a49a-61dde57e69f3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:43.936539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713ec6d-54f9-44fd-a49a-61dde57e69f3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:43.938112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713ec6d-54f9-44fd-a49a-61dde57e69f3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6713ec6d-54f9-44fd-a49a-61dde57e69f3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:43.957720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:43.958928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:43.960105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:43.960105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:43.960105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:43.960105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:43.960105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:43.960105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:43.960105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:43.960105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:43.960105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 07:24:43.960105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:43.964720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 322/664] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:43.965729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713ec6d-54f9-44fd-a49a-61dde57e69f3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:44.003958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 343/665] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:44.005093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96eb1a0-b51e-42e5-9517-491cd14f0de7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:44.090811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713ec6d-54f9-44fd-a49a-61dde57e69f3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:44.122557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:44.137083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a2fb9-579b-45c5-9871-eb8f26d6b8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:44.137281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96eb1a0-b51e-42e5-9517-491cd14f0de7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:44.137639 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a2fb9-579b-45c5-9871-eb8f26d6b8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:44.137926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a2fb9-579b-45c5-9871-eb8f26d6b8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:44.138208 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a2fb9-579b-45c5-9871-eb8f26d6b8f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:44.139007 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a2fb9-579b-45c5-9871-eb8f26d6b8f4 None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=5f57fa108c3d478a9976f4aea5f33bb5) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:44.139364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a2fb9-579b-45c5-9871-eb8f26d6b8f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:44.140824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a2fb9-579b-45c5-9871-eb8f26d6b8f4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:44.147011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 323/666] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:43 2022] GET /identity/v3/role_assignments?user.id=5f57fa108c3d478a9976f4aea5f33bb5&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:44.147852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a2fb9-579b-45c5-9871-eb8f26d6b8f4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:44.179959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f67bdc-5d7c-4571-b04a-32465f8c0722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:44.181920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f67bdc-5d7c-4571-b04a-32465f8c0722 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:44.182544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f67bdc-5d7c-4571-b04a-32465f8c0722 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:44.183105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f67bdc-5d7c-4571-b04a-32465f8c0722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:44.191523 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:44.198525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f67bdc-5d7c-4571-b04a-32465f8c0722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:44.200148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f67bdc-5d7c-4571-b04a-32465f8c0722 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a9f67bdc-5d7c-4571-b04a-32465f8c0722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:44.220602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:44.221843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:44.222930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:44.222930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:44.222930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:44.222930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:44.224333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 344/667] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:44.227353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f67bdc-5d7c-4571-b04a-32465f8c0722 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:44.283883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a2fb9-579b-45c5-9871-eb8f26d6b8f4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:44.327583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37ea95-77b0-4f6f-b0b1-b48a2691c809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:44.328284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37ea95-77b0-4f6f-b0b1-b48a2691c809 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:44.328578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37ea95-77b0-4f6f-b0b1-b48a2691c809 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:44.328956 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37ea95-77b0-4f6f-b0b1-b48a2691c809 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:44.329782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37ea95-77b0-4f6f-b0b1-b48a2691c809 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:44.330096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37ea95-77b0-4f6f-b0b1-b48a2691c809 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:44.331693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37ea95-77b0-4f6f-b0b1-b48a2691c809 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:44.354744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 324/668] 10.222.0.40 () {56 vars in 1139 bytes} [Mon Aug 1 07:24:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:44.355496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37ea95-77b0-4f6f-b0b1-b48a2691c809 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:44.376499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f67bdc-5d7c-4571-b04a-32465f8c0722 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:44.422187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1572808-3f1c-4c26-818a-ba20bd163610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:44.423353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1572808-3f1c-4c26-818a-ba20bd163610 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:44.423779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1572808-3f1c-4c26-818a-ba20bd163610 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:44.424185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1572808-3f1c-4c26-818a-ba20bd163610 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:44.440311 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:44.443778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1572808-3f1c-4c26-818a-ba20bd163610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-d1572808-3f1c-4c26-818a-ba20bd163610 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:44.451922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:44.453085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:44.454027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:44.454027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:44.454027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:44.454027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:44.454027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:44.454027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:44.454027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:44.454027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:44.454027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:44.454027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:44.454027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:44.454027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:44.458002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 345/669] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:44.458918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1572808-3f1c-4c26-818a-ba20bd163610 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:44.477919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:44.491548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37ea95-77b0-4f6f-b0b1-b48a2691c809 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:44.536044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f46c3f-0a7d-47c8-84f4-40c37cc26dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:44.536671 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46c3f-0a7d-47c8-84f4-40c37cc26dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:44.536955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46c3f-0a7d-47c8-84f4-40c37cc26dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:44.537234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46c3f-0a7d-47c8-84f4-40c37cc26dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:44.537927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f46c3f-0a7d-47c8-84f4-40c37cc26dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:44.538238 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f46c3f-0a7d-47c8-84f4-40c37cc26dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:44.539644 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f46c3f-0a7d-47c8-84f4-40c37cc26dc3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:44.552650 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 325/670] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:44.553563 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f46c3f-0a7d-47c8-84f4-40c37cc26dc3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:44.628919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1572808-3f1c-4c26-818a-ba20bd163610 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:44.669555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad48002-45ab-403e-90a7-742a4627035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:44.670190 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad48002-45ab-403e-90a7-742a4627035f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:44.670583 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad48002-45ab-403e-90a7-742a4627035f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:44.670866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad48002-45ab-403e-90a7-742a4627035f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:44.678710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f46c3f-0a7d-47c8-84f4-40c37cc26dc3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:44.723435 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c6e74-4ec1-44dd-af1c-72b8e6d63ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:44.724649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c6e74-4ec1-44dd-af1c-72b8e6d63ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:44.725000 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c6e74-4ec1-44dd-af1c-72b8e6d63ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:44.725290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c6e74-4ec1-44dd-af1c-72b8e6d63ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:44.726119 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c6e74-4ec1-44dd-af1c-72b8e6d63ad6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:44.726610 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c6e74-4ec1-44dd-af1c-72b8e6d63ad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:44.728901 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c6e74-4ec1-44dd-af1c-72b8e6d63ad6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:44.741987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 326/671] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:44.743138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c6e74-4ec1-44dd-af1c-72b8e6d63ad6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:44.762880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:44.787962 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:44.792298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad48002-45ab-403e-90a7-742a4627035f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:44.793945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad48002-45ab-403e-90a7-742a4627035f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:44.877579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c6e74-4ec1-44dd-af1c-72b8e6d63ad6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:44.906221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 346/672] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:44.907990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad48002-45ab-403e-90a7-742a4627035f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:44.912740 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65c5ef-ab67-4123-bb21-60a836ada3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:44.913387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65c5ef-ab67-4123-bb21-60a836ada3cd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:44.913676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65c5ef-ab67-4123-bb21-60a836ada3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:44.913962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65c5ef-ab67-4123-bb21-60a836ada3cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:44.920959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65c5ef-ab67-4123-bb21-60a836ada3cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:44.922486 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65c5ef-ab67-4123-bb21-60a836ada3cd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ca65c5ef-ab67-4123-bb21-60a836ada3cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:44.945809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:44.947455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:44.948432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:44.948432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:44.948432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:44.948432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:44.948432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:44.948432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:44.948432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:44.948432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:44.950943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 327/673] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:44.952062 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65c5ef-ab67-4123-bb21-60a836ada3cd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:45.028443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:45.031567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad48002-45ab-403e-90a7-742a4627035f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:45.043760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:45.070923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb33586-831e-4a2f-b259-a8f0ba5fb74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:45.071653 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb33586-831e-4a2f-b259-a8f0ba5fb74a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:45.071985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb33586-831e-4a2f-b259-a8f0ba5fb74a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:45.072266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb33586-831e-4a2f-b259-a8f0ba5fb74a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:45.080337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65c5ef-ab67-4123-bb21-60a836ada3cd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:45.083630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb33586-831e-4a2f-b259-a8f0ba5fb74a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:45.084763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb33586-831e-4a2f-b259-a8f0ba5fb74a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-4eb33586-831e-4a2f-b259-a8f0ba5fb74a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:45.101108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:45.102039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:45.103014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:45.103014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:45.103014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:45.103014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:45.107042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 347/674] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:45.107945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb33586-831e-4a2f-b259-a8f0ba5fb74a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:45.125877 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605b3c1-04a5-4544-a5ce-c947e97aa8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:45.126540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605b3c1-04a5-4544-a5ce-c947e97aa8da None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:45.126834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605b3c1-04a5-4544-a5ce-c947e97aa8da None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:45.127117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605b3c1-04a5-4544-a5ce-c947e97aa8da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:45.143818 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605b3c1-04a5-4544-a5ce-c947e97aa8da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:45.145531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605b3c1-04a5-4544-a5ce-c947e97aa8da None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7605b3c1-04a5-4544-a5ce-c947e97aa8da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:45.164925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:45.166047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:45.167529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:45.167529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:45.167529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:45.167529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:45.167529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 328/675] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:45.167529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605b3c1-04a5-4544-a5ce-c947e97aa8da None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:45.236575 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb33586-831e-4a2f-b259-a8f0ba5fb74a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:45.280979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330abf2-24fd-4d75-a996-3e51ce9e9993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:45.282467 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330abf2-24fd-4d75-a996-3e51ce9e9993 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:45.283250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330abf2-24fd-4d75-a996-3e51ce9e9993 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:45.283760 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330abf2-24fd-4d75-a996-3e51ce9e9993 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:45.285379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330abf2-24fd-4d75-a996-3e51ce9e9993 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:45.286041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330abf2-24fd-4d75-a996-3e51ce9e9993 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:45.288427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330abf2-24fd-4d75-a996-3e51ce9e9993 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:45.294363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605b3c1-04a5-4544-a5ce-c947e97aa8da None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:45.304914 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 348/676] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:45.306740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330abf2-24fd-4d75-a996-3e51ce9e9993 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:45.324351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:45.333135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:45.336871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dfd55-b604-4ed4-a408-33fe20bf1b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:45.337591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dfd55-b604-4ed4-a408-33fe20bf1b52 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:45.337940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dfd55-b604-4ed4-a408-33fe20bf1b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:45.338323 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dfd55-b604-4ed4-a408-33fe20bf1b52 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:45.339088 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dfd55-b604-4ed4-a408-33fe20bf1b52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:45.339496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dfd55-b604-4ed4-a408-33fe20bf1b52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:45.340973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dfd55-b604-4ed4-a408-33fe20bf1b52 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:45.358107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 329/677] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:45.359217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dfd55-b604-4ed4-a408-33fe20bf1b52 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:45.455322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330abf2-24fd-4d75-a996-3e51ce9e9993 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:45.471414 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dfd55-b604-4ed4-a408-33fe20bf1b52 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:45.511204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7258a9-7178-48ce-9e13-5ed69140ffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:45.511851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7258a9-7178-48ce-9e13-5ed69140ffc2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:45.512200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7258a9-7178-48ce-9e13-5ed69140ffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:45.512541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7258a9-7178-48ce-9e13-5ed69140ffc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:45.513410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7258a9-7178-48ce-9e13-5ed69140ffc2 None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=495c489f00d74b53ac02c2cbbb323c38) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:45.513787 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7258a9-7178-48ce-9e13-5ed69140ffc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:45.515427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7258a9-7178-48ce-9e13-5ed69140ffc2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:45.527481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 330/678] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:45 2022] GET /identity/v3/role_assignments?user.id=495c489f00d74b53ac02c2cbbb323c38&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:45.528754 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7258a9-7178-48ce-9e13-5ed69140ffc2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:45.628855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:45.648042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2183de0-2284-4828-9abf-edfc74a95722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:45.649079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2183de0-2284-4828-9abf-edfc74a95722 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:45.649572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2183de0-2284-4828-9abf-edfc74a95722 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:45.650142 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2183de0-2284-4828-9abf-edfc74a95722 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:45.651002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2183de0-2284-4828-9abf-edfc74a95722 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:45.651481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2183de0-2284-4828-9abf-edfc74a95722 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:45.653481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2183de0-2284-4828-9abf-edfc74a95722 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:45.668686 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 349/679] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:45.676123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2183de0-2284-4828-9abf-edfc74a95722 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:45.709377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7258a9-7178-48ce-9e13-5ed69140ffc2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:45.748337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316db9c-a680-4188-bbd2-df30819c7526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:45.748992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316db9c-a680-4188-bbd2-df30819c7526 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:45.749276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316db9c-a680-4188-bbd2-df30819c7526 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:45.749665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316db9c-a680-4188-bbd2-df30819c7526 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:45.758240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316db9c-a680-4188-bbd2-df30819c7526 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:45.760576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316db9c-a680-4188-bbd2-df30819c7526 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3316db9c-a680-4188-bbd2-df30819c7526 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:45.773438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:45.774990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:45.777204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:45.777204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:45.777204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:45.777204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:45.777204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:45.777204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:45.777204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:45.777204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:45.777204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:24:45.777204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:45.777204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 331/680] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:45.783308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316db9c-a680-4188-bbd2-df30819c7526 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:45.820129 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:45.854061 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2183de0-2284-4828-9abf-edfc74a95722 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:45.890811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316db9c-a680-4188-bbd2-df30819c7526 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:45.892099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b81bf-708c-4835-849b-79923deef48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:45.892974 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b81bf-708c-4835-849b-79923deef48e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:45.893456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b81bf-708c-4835-849b-79923deef48e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:45.893942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b81bf-708c-4835-849b-79923deef48e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:45.898566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:45.903593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b81bf-708c-4835-849b-79923deef48e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:45.905799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b81bf-708c-4835-849b-79923deef48e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-c18b81bf-708c-4835-849b-79923deef48e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:45.918590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:45.919970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:45.920947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:45.920947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:45.920947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:45.920947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:45.920947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:45.920947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:45.920947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:45.920947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:45.923611 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 350/681] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:45.929082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98aa28-d0c8-4a76-9f1d-82bbb5fef3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:45.929082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98aa28-d0c8-4a76-9f1d-82bbb5fef3da None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:45.929082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98aa28-d0c8-4a76-9f1d-82bbb5fef3da None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:45.929082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98aa28-d0c8-4a76-9f1d-82bbb5fef3da None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:45.929590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98aa28-d0c8-4a76-9f1d-82bbb5fef3da None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:45.929962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98aa28-d0c8-4a76-9f1d-82bbb5fef3da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:45.931462 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b81bf-708c-4835-849b-79923deef48e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:45.932928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98aa28-d0c8-4a76-9f1d-82bbb5fef3da None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:45.954507 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 332/682] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Aug 1 07:24:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:45.960173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98aa28-d0c8-4a76-9f1d-82bbb5fef3da None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:46.045250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b81bf-708c-4835-849b-79923deef48e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:46.073794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98aa28-d0c8-4a76-9f1d-82bbb5fef3da None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:46.078046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:46.086372 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a0dea-9120-47e8-88aa-fc1b23159d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:46.087577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a0dea-9120-47e8-88aa-fc1b23159d8c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:46.088090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a0dea-9120-47e8-88aa-fc1b23159d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:46.088371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a0dea-9120-47e8-88aa-fc1b23159d8c None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:46.089135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a0dea-9120-47e8-88aa-fc1b23159d8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:46.089452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a0dea-9120-47e8-88aa-fc1b23159d8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:46.090901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a0dea-9120-47e8-88aa-fc1b23159d8c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:46.103330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 351/683] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:46.104801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a942fc6-6a0f-459a-b857-5faf8de3894c None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:46.105091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a942fc6-6a0f-459a-b857-5faf8de3894c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:46.105367 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a942fc6-6a0f-459a-b857-5faf8de3894c None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:46.106836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 352/684] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:46.112295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a189369-4faa-41e3-a7ad-bb920bd9ccfd None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:46.112479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0443cc4-6c8b-48ae-b08d-811d48747b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:46.112718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a189369-4faa-41e3-a7ad-bb920bd9ccfd None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:46.113050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a189369-4faa-41e3-a7ad-bb920bd9ccfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:46.113156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0443cc4-6c8b-48ae-b08d-811d48747b87 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:46.113447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0443cc4-6c8b-48ae-b08d-811d48747b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:46.113726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0443cc4-6c8b-48ae-b08d-811d48747b87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:46.141361 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5a189369-4faa-41e3-a7ad-bb920bd9ccfd None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:46.143438 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5a189369-4faa-41e3-a7ad-bb920bd9ccfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:46.148459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:46.196529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0443cc4-6c8b-48ae-b08d-811d48747b87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:46.198099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0443cc4-6c8b-48ae-b08d-811d48747b87 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:46.234634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 353/685] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:46.235623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a189369-4faa-41e3-a7ad-bb920bd9ccfd None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:46.317394 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 333/686] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:46.319492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0443cc4-6c8b-48ae-b08d-811d48747b87 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:46.356209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a189369-4faa-41e3-a7ad-bb920bd9ccfd None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:46.388271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:46.396642 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:46.405759 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c1576-33e5-4b63-9edc-16d4f392e80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:46.406892 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c1576-33e5-4b63-9edc-16d4f392e80e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:46.408233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c1576-33e5-4b63-9edc-16d4f392e80e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:46.408659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c1576-33e5-4b63-9edc-16d4f392e80e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:46.409611 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924c1576-33e5-4b63-9edc-16d4f392e80e None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=138ede2854894a3b995435980965911c) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:46.410081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924c1576-33e5-4b63-9edc-16d4f392e80e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:46.411684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924c1576-33e5-4b63-9edc-16d4f392e80e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:46.427098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 354/687] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:46 2022] GET /identity/v3/role_assignments?user.id=138ede2854894a3b995435980965911c&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:46.427098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c1576-33e5-4b63-9edc-16d4f392e80e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:46.458019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0443cc4-6c8b-48ae-b08d-811d48747b87 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:46.499524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a897d-67dc-493a-ae28-5d9937f70ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:46.500318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a897d-67dc-493a-ae28-5d9937f70ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:46.500607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a897d-67dc-493a-ae28-5d9937f70ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:46.501017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a897d-67dc-493a-ae28-5d9937f70ff0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:46.514224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a897d-67dc-493a-ae28-5d9937f70ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-067a897d-67dc-493a-ae28-5d9937f70ff0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:46.519423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:46.520434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:46.521284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:46.521284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:46.521284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:46.521284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:46.521284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:46.521284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:46.521284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:46.521284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:46.521284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:46.521284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:46.521284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:46.521284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:46.525748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 334/688] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:46.527148 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ed8d5-03b7-4984-babb-5e43f6d67401 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:46.527451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ed8d5-03b7-4984-babb-5e43f6d67401 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:46.527739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ed8d5-03b7-4984-babb-5e43f6d67401 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:46.529454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 335/689] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:46.530212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ed8d5-03b7-4984-babb-5e43f6d67401 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:46.553556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c1576-33e5-4b63-9edc-16d4f392e80e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:46.593775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff89a3-84d6-4a4a-bb26-21b2bfe82fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:46.594999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff89a3-84d6-4a4a-bb26-21b2bfe82fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:46.595559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff89a3-84d6-4a4a-bb26-21b2bfe82fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:46.596045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff89a3-84d6-4a4a-bb26-21b2bfe82fd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:46.609152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff89a3-84d6-4a4a-bb26-21b2bfe82fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:46.611115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff89a3-84d6-4a4a-bb26-21b2bfe82fd4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-01ff89a3-84d6-4a4a-bb26-21b2bfe82fd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:46.630054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:46.630992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:46.631926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:46.631926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:46.631926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:46.631926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:46.634494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 355/690] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:46.636534 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a994d-a583-44c5-a598-fc311a3ff424 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:46.637266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a994d-a583-44c5-a598-fc311a3ff424 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:46.637761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a994d-a583-44c5-a598-fc311a3ff424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:46.642776 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ed8d5-03b7-4984-babb-5e43f6d67401 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:46.658686 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-603a994d-a583-44c5-a598-fc311a3ff424 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:46.660281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-603a994d-a583-44c5-a598-fc311a3ff424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:46.669434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:46.682824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531761ef-d27e-496f-9870-7d4bf89ce3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:46.683490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531761ef-d27e-496f-9870-7d4bf89ce3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:46.683778 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531761ef-d27e-496f-9870-7d4bf89ce3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:46.684061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531761ef-d27e-496f-9870-7d4bf89ce3f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:46.684790 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531761ef-d27e-496f-9870-7d4bf89ce3f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:46.685102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531761ef-d27e-496f-9870-7d4bf89ce3f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:46.686487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531761ef-d27e-496f-9870-7d4bf89ce3f9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:46.701343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 336/691] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:46.702229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531761ef-d27e-496f-9870-7d4bf89ce3f9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:46.702706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:46.761444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 356/692] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:46.762930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a994d-a583-44c5-a598-fc311a3ff424 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:46.817663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531761ef-d27e-496f-9870-7d4bf89ce3f9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:46.859734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe2822-78cf-4f53-8f53-e85d20787e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:46.860450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe2822-78cf-4f53-8f53-e85d20787e6b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:46.860885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe2822-78cf-4f53-8f53-e85d20787e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:46.861232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe2822-78cf-4f53-8f53-e85d20787e6b None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:46.862086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe2822-78cf-4f53-8f53-e85d20787e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:46.862457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe2822-78cf-4f53-8f53-e85d20787e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:46.863918 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe2822-78cf-4f53-8f53-e85d20787e6b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:46.878647 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 337/693] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:46.884419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe2822-78cf-4f53-8f53-e85d20787e6b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:46.886071 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a994d-a583-44c5-a598-fc311a3ff424 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:46.931132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad1e75-0b86-40ea-a199-07fe225eedb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:46.931757 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad1e75-0b86-40ea-a199-07fe225eedb8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:46.932558 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad1e75-0b86-40ea-a199-07fe225eedb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:46.932893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad1e75-0b86-40ea-a199-07fe225eedb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:46.940235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:46.948748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad1e75-0b86-40ea-a199-07fe225eedb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:46.950804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad1e75-0b86-40ea-a199-07fe225eedb8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-63ad1e75-0b86-40ea-a199-07fe225eedb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:46.972050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:46.973098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:46.974197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:46.974197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:46.974197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:46.974197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:46.978860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 357/694] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:46.979905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad1e75-0b86-40ea-a199-07fe225eedb8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:47.008459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe2822-78cf-4f53-8f53-e85d20787e6b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:47.009643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:47.050705 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b82ff-f0da-4191-b5b2-954ebfe7fa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:47.051682 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b82ff-f0da-4191-b5b2-954ebfe7fa3c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:47.052179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b82ff-f0da-4191-b5b2-954ebfe7fa3c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:47.052620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b82ff-f0da-4191-b5b2-954ebfe7fa3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:47.063294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b82ff-f0da-4191-b5b2-954ebfe7fa3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:47.064920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b82ff-f0da-4191-b5b2-954ebfe7fa3c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-490b82ff-f0da-4191-b5b2-954ebfe7fa3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:47.080553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:47.081675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:47.082679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:47.082679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:47.082679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:47.082679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:47.084505 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 338/695] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:47.085489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b82ff-f0da-4191-b5b2-954ebfe7fa3c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:47.107561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad1e75-0b86-40ea-a199-07fe225eedb8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:47.145669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deaf37-db55-4ed5-9ab0-3d0d96eb0826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:47.146908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deaf37-db55-4ed5-9ab0-3d0d96eb0826 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:47.147550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deaf37-db55-4ed5-9ab0-3d0d96eb0826 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:47.147796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deaf37-db55-4ed5-9ab0-3d0d96eb0826 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:47.156105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3deaf37-db55-4ed5-9ab0-3d0d96eb0826 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:47.157480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3deaf37-db55-4ed5-9ab0-3d0d96eb0826 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-e3deaf37-db55-4ed5-9ab0-3d0d96eb0826 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:47.172365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:47.173484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:47.174608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:47.174608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:47.174608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:47.174608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:47.174608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:47.174608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:47.174608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:47.174608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:47.180850 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 358/696] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:47.180850 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deaf37-db55-4ed5-9ab0-3d0d96eb0826 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:47.197731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:47.202602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b82ff-f0da-4191-b5b2-954ebfe7fa3c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:47.237831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611890c7-c924-406f-bb5d-5f8dafbba019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:47.238692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611890c7-c924-406f-bb5d-5f8dafbba019 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:47.239182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611890c7-c924-406f-bb5d-5f8dafbba019 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:47.239602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611890c7-c924-406f-bb5d-5f8dafbba019 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:47.240513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611890c7-c924-406f-bb5d-5f8dafbba019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:47.240964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611890c7-c924-406f-bb5d-5f8dafbba019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:47.242581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611890c7-c924-406f-bb5d-5f8dafbba019 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:47.259438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 339/697] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:47.260765 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611890c7-c924-406f-bb5d-5f8dafbba019 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:47.302774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deaf37-db55-4ed5-9ab0-3d0d96eb0826 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:47.310422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:47.340305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce13023-fdf8-437b-8a35-a5bac578a76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:47.341099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce13023-fdf8-437b-8a35-a5bac578a76f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:47.341495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce13023-fdf8-437b-8a35-a5bac578a76f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:47.341885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce13023-fdf8-437b-8a35-a5bac578a76f None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:47.342932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce13023-fdf8-437b-8a35-a5bac578a76f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:47.343377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce13023-fdf8-437b-8a35-a5bac578a76f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:47.344968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce13023-fdf8-437b-8a35-a5bac578a76f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:47.363515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 359/698] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:47.364609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce13023-fdf8-437b-8a35-a5bac578a76f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:47.367150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611890c7-c924-406f-bb5d-5f8dafbba019 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:47.402766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dd199-3278-43b3-a49a-460ae20b174d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:47.403632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dd199-3278-43b3-a49a-460ae20b174d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:47.404103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dd199-3278-43b3-a49a-460ae20b174d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:47.404510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dd199-3278-43b3-a49a-460ae20b174d None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:47.405422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98dd199-3278-43b3-a49a-460ae20b174d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:47.405855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98dd199-3278-43b3-a49a-460ae20b174d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:47.407440 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98dd199-3278-43b3-a49a-460ae20b174d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:47.425003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 340/699] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:47.426014 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dd199-3278-43b3-a49a-460ae20b174d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:47.445460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:47.481683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce13023-fdf8-437b-8a35-a5bac578a76f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:47.519661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652b3ba-5fed-407c-bb8d-140f3a4dd064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:47.520495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652b3ba-5fed-407c-bb8d-140f3a4dd064 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:47.520939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652b3ba-5fed-407c-bb8d-140f3a4dd064 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:47.521328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652b3ba-5fed-407c-bb8d-140f3a4dd064 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:47.530296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652b3ba-5fed-407c-bb8d-140f3a4dd064 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:47.531596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dd199-3278-43b3-a49a-460ae20b174d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:47.532041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652b3ba-5fed-407c-bb8d-140f3a4dd064 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:47.535476 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9652b3ba-5fed-407c-bb8d-140f3a4dd064 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:47.548479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:47.549663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:47.550905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:47.550905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:47.550905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:47.550905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:47.550905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:47.550905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:47.550905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:47.550905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:47.550905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:24:47.550905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:47.558699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 360/700] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Aug 1 07:24:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:47.558699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652b3ba-5fed-407c-bb8d-140f3a4dd064 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:47.577890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd34529-5138-4038-a837-8e290339048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:47.578843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd34529-5138-4038-a837-8e290339048d None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:47.580408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd34529-5138-4038-a837-8e290339048d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:47.580911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd34529-5138-4038-a837-8e290339048d None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:47.581811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd34529-5138-4038-a837-8e290339048d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:47.583515 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd34529-5138-4038-a837-8e290339048d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:47.615664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 341/701] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:24:47 2022] POST /identity/v3/users => generated 254 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:24:47.617172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd34529-5138-4038-a837-8e290339048d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:47.708876 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652b3ba-5fed-407c-bb8d-140f3a4dd064 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:47.782695 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c637f-0031-43be-9526-d7db74bac65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:47.783366 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c637f-0031-43be-9526-d7db74bac65e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:47.783656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c637f-0031-43be-9526-d7db74bac65e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:47.783937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c637f-0031-43be-9526-d7db74bac65e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:47.784740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c637f-0031-43be-9526-d7db74bac65e None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=2ba2655c2eef408498c0298d432263f6) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:47.785056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c637f-0031-43be-9526-d7db74bac65e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:47.787174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c637f-0031-43be-9526-d7db74bac65e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:47.798177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 361/702] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:47 2022] GET /identity/v3/role_assignments?user.id=2ba2655c2eef408498c0298d432263f6&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:47.815914 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:47.845319 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd34529-5138-4038-a837-8e290339048d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:47.886349 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07a3f2-ec86-42b6-89f0-ed96771902e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:47.887008 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07a3f2-ec86-42b6-89f0-ed96771902e1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:47.887337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07a3f2-ec86-42b6-89f0-ed96771902e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:47.887616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07a3f2-ec86-42b6-89f0-ed96771902e1 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:47.888433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07a3f2-ec86-42b6-89f0-ed96771902e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:47.888747 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07a3f2-ec86-42b6-89f0-ed96771902e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:47.890185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07a3f2-ec86-42b6-89f0-ed96771902e1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:47.910562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 342/703] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Aug 1 07:24:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:47.916981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c637f-0031-43be-9526-d7db74bac65e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:48.006675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:48.048048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c637f-0031-43be-9526-d7db74bac65e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:48.085372 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e80a53-c8a5-40c3-b91c-69e0988a1d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:48.086005 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e80a53-c8a5-40c3-b91c-69e0988a1d23 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:48.086288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e80a53-c8a5-40c3-b91c-69e0988a1d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:48.086601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e80a53-c8a5-40c3-b91c-69e0988a1d23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:48.113620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b6b0d-8441-4c60-a26e-67443c6f51d6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:48.113909 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b6b0d-8441-4c60-a26e-67443c6f51d6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:48.114188 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b6b0d-8441-4c60-a26e-67443c6f51d6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:48.115701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 343/704] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:24:48.122209 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a1919-5f16-467e-b17e-c98d88091bc8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:48.122666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a1919-5f16-467e-b17e-c98d88091bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:48.122951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a1919-5f16-467e-b17e-c98d88091bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:48.148968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ee6a1919-5f16-467e-b17e-c98d88091bc8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:48.150769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ee6a1919-5f16-467e-b17e-c98d88091bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:48.165300 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e80a53-c8a5-40c3-b91c-69e0988a1d23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:48.166942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e80a53-c8a5-40c3-b91c-69e0988a1d23 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:48.244002 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 344/705] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:48.253508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:48.276409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a1919-5f16-467e-b17e-c98d88091bc8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:48.326146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 362/706] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:48.329866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e80a53-c8a5-40c3-b91c-69e0988a1d23 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:48.403442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:48.472679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a1919-5f16-467e-b17e-c98d88091bc8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:48.500806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e80a53-c8a5-40c3-b91c-69e0988a1d23 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:48.544577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ee0ff-a787-4ff6-a020-fd76d55da0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:48.545598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ee0ff-a787-4ff6-a020-fd76d55da0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:48.547525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ee0ff-a787-4ff6-a020-fd76d55da0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:48.548023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ee0ff-a787-4ff6-a020-fd76d55da0e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:48.568874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ee0ff-a787-4ff6-a020-fd76d55da0e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:48.571750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ee0ff-a787-4ff6-a020-fd76d55da0e7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:48.572651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e5370-8af8-4fb0-be2d-d86565147b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:48.573554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e5370-8af8-4fb0-be2d-d86565147b2f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:48.574054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e5370-8af8-4fb0-be2d-d86565147b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:48.574539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e5370-8af8-4fb0-be2d-d86565147b2f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:48.575412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e5370-8af8-4fb0-be2d-d86565147b2f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:48.577449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e5370-8af8-4fb0-be2d-d86565147b2f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-367e5370-8af8-4fb0-be2d-d86565147b2f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:48.598468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:48.599537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:24:48.600374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-f88ee0ff-a787-4ff6-a020-fd76d55da0e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:48.608343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:48.609222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:48.610118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:48.610118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:48.610118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:48.610118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:48.613304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 345/707] 10.222.0.40 () {56 vars in 1129 bytes} [Mon Aug 1 07:24:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:48.614934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 363/708] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:48.620937 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e5370-8af8-4fb0-be2d-d86565147b2f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:48.622969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ee0ff-a787-4ff6-a020-fd76d55da0e7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:48.668844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:48.734953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e5370-8af8-4fb0-be2d-d86565147b2f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:48.763454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:48.772981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3112cd-7cb0-4218-b64d-c3345ca18fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:48.773993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3112cd-7cb0-4218-b64d-c3345ca18fcd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:48.775187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3112cd-7cb0-4218-b64d-c3345ca18fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:48.775895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3112cd-7cb0-4218-b64d-c3345ca18fcd None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:48.776819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3112cd-7cb0-4218-b64d-c3345ca18fcd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:48.777343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3112cd-7cb0-4218-b64d-c3345ca18fcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:48.778944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3112cd-7cb0-4218-b64d-c3345ca18fcd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:48.785256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 346/709] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Aug 1 07:24:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:48.787428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f494e6-d568-40e0-8ab8-1b915794522c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:48.788143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f494e6-d568-40e0-8ab8-1b915794522c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:48.790513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f494e6-d568-40e0-8ab8-1b915794522c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:48.792001 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 347/710] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:48.795091 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f494e6-d568-40e0-8ab8-1b915794522c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:48.822148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ee0ff-a787-4ff6-a020-fd76d55da0e7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:48.858505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a38e8a-1a73-463e-8cb8-9c5300cc2b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:48.859366 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a38e8a-1a73-463e-8cb8-9c5300cc2b65 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:48.859784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a38e8a-1a73-463e-8cb8-9c5300cc2b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:48.860278 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a38e8a-1a73-463e-8cb8-9c5300cc2b65 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:48.861021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a38e8a-1a73-463e-8cb8-9c5300cc2b65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:48.861707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a38e8a-1a73-463e-8cb8-9c5300cc2b65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:48.862966 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a38e8a-1a73-463e-8cb8-9c5300cc2b65 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:48.876507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 364/711] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:48.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3472db6-5813-4c37-ab7e-d3662a382801 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:48.878232 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3472db6-5813-4c37-ab7e-d3662a382801 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:48.878629 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3472db6-5813-4c37-ab7e-d3662a382801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:48.897542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c3472db6-5813-4c37-ab7e-d3662a382801 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:48.898901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c3472db6-5813-4c37-ab7e-d3662a382801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:48.938501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f494e6-d568-40e0-8ab8-1b915794522c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:48.984485 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 365/712] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:48.989796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5fd42-0bae-49db-81ee-35322d22ce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:48.989796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5fd42-0bae-49db-81ee-35322d22ce2f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:48.989796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5fd42-0bae-49db-81ee-35322d22ce2f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:48.989796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5fd42-0bae-49db-81ee-35322d22ce2f None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:48.990173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5fd42-0bae-49db-81ee-35322d22ce2f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:48.990173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5fd42-0bae-49db-81ee-35322d22ce2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:48.990173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5fd42-0bae-49db-81ee-35322d22ce2f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:48.991293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3472db6-5813-4c37-ab7e-d3662a382801 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:49.004108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 348/713] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Aug 1 07:24:48 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:49.010357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:49.047144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5fd42-0bae-49db-81ee-35322d22ce2f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:49.140903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:49.173213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3472db6-5813-4c37-ab7e-d3662a382801 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:49.195149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5fd42-0bae-49db-81ee-35322d22ce2f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:49.237466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c6176-0032-4205-acc7-7b9aa7ba21e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:49.238536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c6176-0032-4205-acc7-7b9aa7ba21e3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:49.239033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c6176-0032-4205-acc7-7b9aa7ba21e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:49.239542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c6176-0032-4205-acc7-7b9aa7ba21e3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:49.240505 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c6176-0032-4205-acc7-7b9aa7ba21e3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:49.242321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c6176-0032-4205-acc7-7b9aa7ba21e3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:49.245447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94875cf1-0db9-4802-86f7-dd13f26420f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:49.246161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94875cf1-0db9-4802-86f7-dd13f26420f2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:49.246450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94875cf1-0db9-4802-86f7-dd13f26420f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:49.246739 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94875cf1-0db9-4802-86f7-dd13f26420f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7d6c6176-0032-4205-acc7-7b9aa7ba21e3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:49.254250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:49.255400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:24:49.256239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:49.256994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94875cf1-0db9-4802-86f7-dd13f26420f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:49.256994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94875cf1-0db9-4802-86f7-dd13f26420f2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:49.259160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 349/714] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:49 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:49.266064 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c6176-0032-4205-acc7-7b9aa7ba21e3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-94875cf1-0db9-4802-86f7-dd13f26420f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:49.272440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:49.273479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:49.274834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:49.274834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:49.274834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:49.274834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:49.274834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:49.274834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:49.274834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:49.274834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:49.276566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 366/715] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:49.281739 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94875cf1-0db9-4802-86f7-dd13f26420f2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:49.358324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:49.377015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c6176-0032-4205-acc7-7b9aa7ba21e3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:49.395077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:49.400284 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94875cf1-0db9-4802-86f7-dd13f26420f2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:49.413335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e3020-68c3-43e4-8205-0d6dd88e7542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:49.414156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e3020-68c3-43e4-8205-0d6dd88e7542 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:49.415475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e3020-68c3-43e4-8205-0d6dd88e7542 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:49.416006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e3020-68c3-43e4-8205-0d6dd88e7542 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:49.416869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e3020-68c3-43e4-8205-0d6dd88e7542 None admin] RBAC: Adding query filter params (name=network) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:49.417361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e3020-68c3-43e4-8205-0d6dd88e7542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:49.418925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e3020-68c3-43e4-8205-0d6dd88e7542 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:49.424308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 350/716] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:24:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:49.429839 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e3020-68c3-43e4-8205-0d6dd88e7542 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:49.439949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b0f32-bab9-45b9-9362-4027f9f8a7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:49.440466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b0f32-bab9-45b9-9362-4027f9f8a7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:49.440753 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b0f32-bab9-45b9-9362-4027f9f8a7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:49.441033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b0f32-bab9-45b9-9362-4027f9f8a7d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:49.441987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b0f32-bab9-45b9-9362-4027f9f8a7d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:49.442228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b0f32-bab9-45b9-9362-4027f9f8a7d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:49.443837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b0f32-bab9-45b9-9362-4027f9f8a7d6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:49.459667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 367/717] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:49.465869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b0f32-bab9-45b9-9362-4027f9f8a7d6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:49.542659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e3020-68c3-43e4-8205-0d6dd88e7542 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:49.596792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b0f32-bab9-45b9-9362-4027f9f8a7d6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:49.609253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b687fdd-42c7-4984-b969-6bfcbe5ee4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:49.610176 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b687fdd-42c7-4984-b969-6bfcbe5ee4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:49.610680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b687fdd-42c7-4984-b969-6bfcbe5ee4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:49.611190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b687fdd-42c7-4984-b969-6bfcbe5ee4c9 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:49.612076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b687fdd-42c7-4984-b969-6bfcbe5ee4c9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:49.612587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b687fdd-42c7-4984-b969-6bfcbe5ee4c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:49.614223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b687fdd-42c7-4984-b969-6bfcbe5ee4c9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:49.620360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 351/718] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:24:49 2022] GET /identity/v3/services?type=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:49.641809 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:49.653523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daa56f-1e5b-4f6a-b0e6-af692a2b1f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:49.654522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daa56f-1e5b-4f6a-b0e6-af692a2b1f77 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:49.655011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daa56f-1e5b-4f6a-b0e6-af692a2b1f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:49.655524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daa56f-1e5b-4f6a-b0e6-af692a2b1f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:49.656535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daa56f-1e5b-4f6a-b0e6-af692a2b1f77 None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=abdcd6d09fee45c696449641ea36eae3) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:49.657070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daa56f-1e5b-4f6a-b0e6-af692a2b1f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:49.658777 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daa56f-1e5b-4f6a-b0e6-af692a2b1f77 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:49.666926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 368/719] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:49 2022] GET /identity/v3/role_assignments?user.id=abdcd6d09fee45c696449641ea36eae3&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:50.144663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba3325-b584-4779-a27b-5e0bd4945dd4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:50.145198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba3325-b584-4779-a27b-5e0bd4945dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:50.145198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba3325-b584-4779-a27b-5e0bd4945dd4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:50.146830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 352/720] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:50.152669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda44f7-470b-4ec3-9857-01e6465d21a0 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:50.153159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda44f7-470b-4ec3-9857-01e6465d21a0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:50.153268 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda44f7-470b-4ec3-9857-01e6465d21a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:50.180680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8fda44f7-470b-4ec3-9857-01e6465d21a0 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:50.181888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8fda44f7-470b-4ec3-9857-01e6465d21a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:50.278848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 369/721] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:50.301638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba3325-b584-4779-a27b-5e0bd4945dd4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:50.351883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:50.402692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551e81d-2bc4-45ca-be69-0698ca7c062c None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:50.403015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551e81d-2bc4-45ca-be69-0698ca7c062c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:50.403801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551e81d-2bc4-45ca-be69-0698ca7c062c None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:50.405393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 370/722] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:24:50.409995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7f961-944a-415c-9077-752d5c243af4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:50.410289 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7f961-944a-415c-9077-752d5c243af4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:50.410571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7f961-944a-415c-9077-752d5c243af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:50.423353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba3325-b584-4779-a27b-5e0bd4945dd4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:50.428801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5fe7f961-944a-415c-9077-752d5c243af4 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:50.430685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5fe7f961-944a-415c-9077-752d5c243af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:50.479386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2cbed-f2a1-491a-85e2-0123fb3e3c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:50.480408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2cbed-f2a1-491a-85e2-0123fb3e3c10 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:50.480899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2cbed-f2a1-491a-85e2-0123fb3e3c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:50.481373 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2cbed-f2a1-491a-85e2-0123fb3e3c10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:50.496016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2cbed-f2a1-491a-85e2-0123fb3e3c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-88c2cbed-f2a1-491a-85e2-0123fb3e3c10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:50.501681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:50.502585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:50.503461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:50.503461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:50.503461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:50.503461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:50.503461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:50.503461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:50.503461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:50.503461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:50.503461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:50.503461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:50.503461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:50.503461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:50.510866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 353/723] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:50.513701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2cbed-f2a1-491a-85e2-0123fb3e3c10 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:50.553332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 371/724] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 07:24:50.591779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7f961-944a-415c-9077-752d5c243af4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:50.625992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2cbed-f2a1-491a-85e2-0123fb3e3c10 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:50.633976 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:50.670350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7e8a6-ea2b-480e-8c6e-6d905b0d4251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:50.670978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7e8a6-ea2b-480e-8c6e-6d905b0d4251 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:50.671282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7e8a6-ea2b-480e-8c6e-6d905b0d4251 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:50.671558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7e8a6-ea2b-480e-8c6e-6d905b0d4251 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:50.672273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7e8a6-ea2b-480e-8c6e-6d905b0d4251 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:50.672584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7e8a6-ea2b-480e-8c6e-6d905b0d4251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:50.673953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7e8a6-ea2b-480e-8c6e-6d905b0d4251 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:50.691422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 354/725] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:50.701114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7e8a6-ea2b-480e-8c6e-6d905b0d4251 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:50.709599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:50.770036 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7f961-944a-415c-9077-752d5c243af4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:50.812325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b88612-abec-4911-8f71-824f61bde5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:50.812898 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b88612-abec-4911-8f71-824f61bde5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:50.813176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b88612-abec-4911-8f71-824f61bde5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:50.813449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b88612-abec-4911-8f71-824f61bde5c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:50.839994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b88612-abec-4911-8f71-824f61bde5c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-70b88612-abec-4911-8f71-824f61bde5c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:50.845310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:50.846305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:50.847182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:50.847182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:50.847182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:50.847182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:50.847182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:50.847182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:50.847182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:50.847182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:50.847182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:50.847182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:50.847182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:50.847182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:50.856555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 372/726] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:50.862489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b88612-abec-4911-8f71-824f61bde5c7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:50.881645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7e8a6-ea2b-480e-8c6e-6d905b0d4251 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:50.928273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634c541-25a9-4b00-8b0d-c7e7608adae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:50.928921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634c541-25a9-4b00-8b0d-c7e7608adae4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:50.929204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634c541-25a9-4b00-8b0d-c7e7608adae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:50.929477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634c541-25a9-4b00-8b0d-c7e7608adae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:50.953663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634c541-25a9-4b00-8b0d-c7e7608adae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:50.955203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634c541-25a9-4b00-8b0d-c7e7608adae4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7634c541-25a9-4b00-8b0d-c7e7608adae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:50.987301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:50.988248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:50.995003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:50.995003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:50.995003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:50.995003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:50.995003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 355/727] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:51.004107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b88612-abec-4911-8f71-824f61bde5c7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:51.008044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634c541-25a9-4b00-8b0d-c7e7608adae4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:51.017275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:51.040218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:51.055847 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd362f88-1367-460e-b536-90f26ac7e947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:51.056473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd362f88-1367-460e-b536-90f26ac7e947 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:51.056959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd362f88-1367-460e-b536-90f26ac7e947 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:51.058259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd362f88-1367-460e-b536-90f26ac7e947 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:51.058802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd362f88-1367-460e-b536-90f26ac7e947 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:51.059898 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd362f88-1367-460e-b536-90f26ac7e947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:51.061511 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd362f88-1367-460e-b536-90f26ac7e947 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:51.081110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 373/728] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:51.087359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd362f88-1367-460e-b536-90f26ac7e947 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:51.177443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634c541-25a9-4b00-8b0d-c7e7608adae4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:51.199995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd362f88-1367-460e-b536-90f26ac7e947 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:51.241247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbad454-40fa-41bf-9ff8-75a010f100df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:51.241900 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbad454-40fa-41bf-9ff8-75a010f100df None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:51.242187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbad454-40fa-41bf-9ff8-75a010f100df None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:51.242471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbad454-40fa-41bf-9ff8-75a010f100df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:51.246939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947e83f-2ee4-41a3-8509-250bf2492ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:51.247887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947e83f-2ee4-41a3-8509-250bf2492ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:51.248391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947e83f-2ee4-41a3-8509-250bf2492ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:51.248887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947e83f-2ee4-41a3-8509-250bf2492ef5 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:51.249795 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947e83f-2ee4-41a3-8509-250bf2492ef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:51.250301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947e83f-2ee4-41a3-8509-250bf2492ef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:51.257105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947e83f-2ee4-41a3-8509-250bf2492ef5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:51.269880 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbad454-40fa-41bf-9ff8-75a010f100df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:51.271684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbad454-40fa-41bf-9ff8-75a010f100df None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:51.283347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 356/729] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:51.292039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947e83f-2ee4-41a3-8509-250bf2492ef5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-1bbad454-40fa-41bf-9ff8-75a010f100df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:51.300948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:51.301865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:51.302771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:51.302771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:51.302771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:51.302771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:51.302771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 374/730] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:51.308109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbad454-40fa-41bf-9ff8-75a010f100df None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:51.361465 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:51.409364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:51.433561 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947e83f-2ee4-41a3-8509-250bf2492ef5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:51.490894 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e70ae-490e-46a5-9ecb-1c034848d34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:51.491567 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e70ae-490e-46a5-9ecb-1c034848d34f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:51.491853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e70ae-490e-46a5-9ecb-1c034848d34f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:51.492126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e70ae-490e-46a5-9ecb-1c034848d34f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:51.503583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e70ae-490e-46a5-9ecb-1c034848d34f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:51.506345 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e70ae-490e-46a5-9ecb-1c034848d34f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:51.519641 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbad454-40fa-41bf-9ff8-75a010f100df None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3b0e70ae-490e-46a5-9ecb-1c034848d34f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:51.527437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:51.528377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:51.529315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:51.529315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:51.529315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:51.529315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:51.529315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:51.529315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:51.529315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:51.529315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:51.529315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:24:51.529315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:51.533250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 357/731] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Aug 1 07:24:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:51.544054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e70ae-490e-46a5-9ecb-1c034848d34f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:51.576857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc5414-f2a3-4c78-93c9-13e6843dfe0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:51.577729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc5414-f2a3-4c78-93c9-13e6843dfe0d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:51.578188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc5414-f2a3-4c78-93c9-13e6843dfe0d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:51.578622 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc5414-f2a3-4c78-93c9-13e6843dfe0d None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:51.579629 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc5414-f2a3-4c78-93c9-13e6843dfe0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:51.580102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc5414-f2a3-4c78-93c9-13e6843dfe0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:51.581702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc5414-f2a3-4c78-93c9-13e6843dfe0d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:51.607568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 375/732] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:51.609196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dd1c9-6945-4db3-ae79-93da11b80409 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:51.609632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dd1c9-6945-4db3-ae79-93da11b80409 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:51.610082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dd1c9-6945-4db3-ae79-93da11b80409 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:51.613021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 376/733] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:51.614012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dd1c9-6945-4db3-ae79-93da11b80409 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:51.649728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e70ae-490e-46a5-9ecb-1c034848d34f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:51.687401 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecf7b6-c22d-4ee7-a977-0b7813668098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:51.688175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecf7b6-c22d-4ee7-a977-0b7813668098 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:51.688653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecf7b6-c22d-4ee7-a977-0b7813668098 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:51.689033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecf7b6-c22d-4ee7-a977-0b7813668098 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:51.690008 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecf7b6-c22d-4ee7-a977-0b7813668098 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:51.690439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecf7b6-c22d-4ee7-a977-0b7813668098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:51.692003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecf7b6-c22d-4ee7-a977-0b7813668098 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:51.702632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:51.712476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 358/734] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Aug 1 07:24:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:51.713925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0e472-d1d7-4d5b-a935-05a33873bd02 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:51.714213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0e472-d1d7-4d5b-a935-05a33873bd02 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:51.714490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0e472-d1d7-4d5b-a935-05a33873bd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:51.715189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:51.741410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dd1c9-6945-4db3-ae79-93da11b80409 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:51.741760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6dd0e472-d1d7-4d5b-a935-05a33873bd02 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:51.743531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6dd0e472-d1d7-4d5b-a935-05a33873bd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:51.780027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9fef8-07ad-4165-a061-3818fe8d9397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:51.781075 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9fef8-07ad-4165-a061-3818fe8d9397 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:51.781572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9fef8-07ad-4165-a061-3818fe8d9397 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:51.782149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9fef8-07ad-4165-a061-3818fe8d9397 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:51.790945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9fef8-07ad-4165-a061-3818fe8d9397 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:51.792744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9fef8-07ad-4165-a061-3818fe8d9397 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-13c9fef8-07ad-4165-a061-3818fe8d9397 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:51.805559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:51.806578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:51.807547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:51.807547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:51.807547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:51.807547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:51.807547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:51.807547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:51.807547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:51.807547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:51.807547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:24:51.807547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:51.815502 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 377/735] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:51.816143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9fef8-07ad-4165-a061-3818fe8d9397 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:51.819201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 359/736] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:51.819982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0e472-d1d7-4d5b-a935-05a33873bd02 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:51.924446 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9fef8-07ad-4165-a061-3818fe8d9397 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:51.928030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0e472-d1d7-4d5b-a935-05a33873bd02 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:51.959982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7db09a-f64d-41f5-a117-04cde89ba3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:51.961237 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db09a-f64d-41f5-a117-04cde89ba3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:51.962332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db09a-f64d-41f5-a117-04cde89ba3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:51.962983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7db09a-f64d-41f5-a117-04cde89ba3c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:51.963466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:51.975102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf0f66-3348-4389-9b57-71a0a738a3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:51.975102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf0f66-3348-4389-9b57-71a0a738a3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:51.975102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf0f66-3348-4389-9b57-71a0a738a3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:51.975102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf0f66-3348-4389-9b57-71a0a738a3d2 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:51.975524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf0f66-3348-4389-9b57-71a0a738a3d2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:51.975524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf0f66-3348-4389-9b57-71a0a738a3d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:51.975524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf0f66-3348-4389-9b57-71a0a738a3d2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:51.979970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:52.043290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7db09a-f64d-41f5-a117-04cde89ba3c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:52.044798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7db09a-f64d-41f5-a117-04cde89ba3c6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:52.119128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 360/737] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Aug 1 07:24:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:52.120307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf0f66-3348-4389-9b57-71a0a738a3d2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:52.167197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 378/738] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:52.168210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7db09a-f64d-41f5-a117-04cde89ba3c6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:52.219857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:52.227972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf0f66-3348-4389-9b57-71a0a738a3d2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:52.263564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac1693-90ed-4c74-be36-77b6b949dbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:52.264566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac1693-90ed-4c74-be36-77b6b949dbb2 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:52.264839 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac1693-90ed-4c74-be36-77b6b949dbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:52.265120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac1693-90ed-4c74-be36-77b6b949dbb2 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:52.265806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac1693-90ed-4c74-be36-77b6b949dbb2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:52.267189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac1693-90ed-4c74-be36-77b6b949dbb2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:52.282164 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7db09a-f64d-41f5-a117-04cde89ba3c6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:52.284198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 361/739] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:24:52 2022] POST /identity/v3/services => generated 251 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:24:52.285021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac1693-90ed-4c74-be36-77b6b949dbb2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:52.329388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480daca6-f5be-4106-a877-1c85c4e5d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:52.330204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480daca6-f5be-4106-a877-1c85c4e5d417 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:52.330412 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480daca6-f5be-4106-a877-1c85c4e5d417 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:52.330685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480daca6-f5be-4106-a877-1c85c4e5d417 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:52.343989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:52.406858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac1693-90ed-4c74-be36-77b6b949dbb2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:52.414040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480daca6-f5be-4106-a877-1c85c4e5d417 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:52.416273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480daca6-f5be-4106-a877-1c85c4e5d417 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:52.448489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48345690-6627-4d8d-a5a6-fc7aa7a0ef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:52.449135 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48345690-6627-4d8d-a5a6-fc7aa7a0ef03 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:52.449426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48345690-6627-4d8d-a5a6-fc7aa7a0ef03 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:52.449709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48345690-6627-4d8d-a5a6-fc7aa7a0ef03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:52.460288 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48345690-6627-4d8d-a5a6-fc7aa7a0ef03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:52.462424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48345690-6627-4d8d-a5a6-fc7aa7a0ef03 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:52.465999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-48345690-6627-4d8d-a5a6-fc7aa7a0ef03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:52.479094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:52.480004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:52.480895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:52.480895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:52.480895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:52.480895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:52.483183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 362/740] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:52.484740 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d7737-708d-417c-9a7e-ac14d5f80f6b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:52.485277 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d7737-708d-417c-9a7e-ac14d5f80f6b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:52.485591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d7737-708d-417c-9a7e-ac14d5f80f6b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:52.487016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 363/741] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:52.487907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15e8b2-6e23-4786-90a0-a3e1b4b366fe None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:52.488184 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15e8b2-6e23-4786-90a0-a3e1b4b366fe None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:52.488451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15e8b2-6e23-4786-90a0-a3e1b4b366fe None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:52.489753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 364/742] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:52.490289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15e8b2-6e23-4786-90a0-a3e1b4b366fe None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:52.532663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 379/743] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:24:52 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 07:24:52.534352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de69db-c290-4fe4-ba4a-dde7ccd062fa None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:52.534846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de69db-c290-4fe4-ba4a-dde7ccd062fa None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:52.535433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de69db-c290-4fe4-ba4a-dde7ccd062fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:52.553725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f3de69db-c290-4fe4-ba4a-dde7ccd062fa None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:52.555945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f3de69db-c290-4fe4-ba4a-dde7ccd062fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:52.599486 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:52.605723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15e8b2-6e23-4786-90a0-a3e1b4b366fe None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:52.639368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 380/744] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:52 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:52.640570 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c61b41-ea94-43ef-afc1-8bcc0f0137c8 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:52.640906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c61b41-ea94-43ef-afc1-8bcc0f0137c8 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:52.641145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c61b41-ea94-43ef-afc1-8bcc0f0137c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:52.652369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7325aa-7932-4588-8aae-381bdfaec67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:52.653052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7325aa-7932-4588-8aae-381bdfaec67a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:52.653363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7325aa-7932-4588-8aae-381bdfaec67a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:52.653675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7325aa-7932-4588-8aae-381bdfaec67a None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:52.654736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7325aa-7932-4588-8aae-381bdfaec67a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:52.654900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7325aa-7932-4588-8aae-381bdfaec67a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:52.656325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7325aa-7932-4588-8aae-381bdfaec67a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:52.668093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b9c61b41-ea94-43ef-afc1-8bcc0f0137c8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:52.669309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 365/745] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:52.669722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b9c61b41-ea94-43ef-afc1-8bcc0f0137c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:52.670213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7325aa-7932-4588-8aae-381bdfaec67a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:52.737944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:52.751421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 381/746] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:52 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 07:24:52.752398 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c61b41-ea94-43ef-afc1-8bcc0f0137c8 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:52.780950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7325aa-7932-4588-8aae-381bdfaec67a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:52.816685 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65b456-5988-49c6-94f8-70599f89e286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:52.817329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65b456-5988-49c6-94f8-70599f89e286 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:52.817617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65b456-5988-49c6-94f8-70599f89e286 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:52.817901 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65b456-5988-49c6-94f8-70599f89e286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:52.830102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65b456-5988-49c6-94f8-70599f89e286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:52.831592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65b456-5988-49c6-94f8-70599f89e286 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-de65b456-5988-49c6-94f8-70599f89e286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:52.847947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:52.848870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:52.849753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:52.849753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:52.849753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:52.849753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:52.849753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 366/747] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:52.850033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65b456-5988-49c6-94f8-70599f89e286 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:52.866212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:52.874901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c61b41-ea94-43ef-afc1-8bcc0f0137c8 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:52.911035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977cacb-3642-46ed-9f8d-35fa415ce4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:52.911703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977cacb-3642-46ed-9f8d-35fa415ce4ec None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:52.911991 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977cacb-3642-46ed-9f8d-35fa415ce4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:52.912696 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977cacb-3642-46ed-9f8d-35fa415ce4ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:52.926381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977cacb-3642-46ed-9f8d-35fa415ce4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-1977cacb-3642-46ed-9f8d-35fa415ce4ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:52.932934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:52.933860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:52.934698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:52.934698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:52.934698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:52.934698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:52.934698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:52.934698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:52.934698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:52.934698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:52.934698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:52.934698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:52.934698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:52.934698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:52.936070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 382/748] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:52.936070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977cacb-3642-46ed-9f8d-35fa415ce4ec None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:52.967529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65b456-5988-49c6-94f8-70599f89e286 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:53.004390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bf781-f9ac-489d-abee-b80768f85ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:53.005468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bf781-f9ac-489d-abee-b80768f85ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:53.005908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bf781-f9ac-489d-abee-b80768f85ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:53.006340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bf781-f9ac-489d-abee-b80768f85ba5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:53.007776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:53.016133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bf781-f9ac-489d-abee-b80768f85ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:53.017966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bf781-f9ac-489d-abee-b80768f85ba5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-771bf781-f9ac-489d-abee-b80768f85ba5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:53.031289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:53.032403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:53.033526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:53.033526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:53.033526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:53.033526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:53.033526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:53.033526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:53.033526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:53.033526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:53.040327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 367/749] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:53.040327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bf781-f9ac-489d-abee-b80768f85ba5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:53.052638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977cacb-3642-46ed-9f8d-35fa415ce4ec None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:53.089855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d242e-ad1c-4829-bc60-38905cfde10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:53.090475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d242e-ad1c-4829-bc60-38905cfde10d None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:53.090756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d242e-ad1c-4829-bc60-38905cfde10d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:53.091031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d242e-ad1c-4829-bc60-38905cfde10d None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:53.091747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d242e-ad1c-4829-bc60-38905cfde10d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:53.093122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d242e-ad1c-4829-bc60-38905cfde10d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:53.111732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 383/750] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:24:52 2022] POST /identity/v3/services => generated 240 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:24:53.112142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:53.112542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d242e-ad1c-4829-bc60-38905cfde10d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:53.158981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bf781-f9ac-489d-abee-b80768f85ba5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:53.214787 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6808970-bb1a-4d63-9799-b049ef8bb7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:53.215835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6808970-bb1a-4d63-9799-b049ef8bb7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:53.216328 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6808970-bb1a-4d63-9799-b049ef8bb7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:53.216879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6808970-bb1a-4d63-9799-b049ef8bb7f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:53.217792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6808970-bb1a-4d63-9799-b049ef8bb7f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:53.218324 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6808970-bb1a-4d63-9799-b049ef8bb7f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:53.219982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6808970-bb1a-4d63-9799-b049ef8bb7f7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:53.232182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d242e-ad1c-4829-bc60-38905cfde10d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:53.234446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 368/751] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:53.235956 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6808970-bb1a-4d63-9799-b049ef8bb7f7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:53.260919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:53.278967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcae07f-bcf1-43ac-b08e-b83f1fd0a882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:53.279640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcae07f-bcf1-43ac-b08e-b83f1fd0a882 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:53.279926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcae07f-bcf1-43ac-b08e-b83f1fd0a882 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:53.280209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcae07f-bcf1-43ac-b08e-b83f1fd0a882 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:53.280934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcae07f-bcf1-43ac-b08e-b83f1fd0a882 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:53.281246 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcae07f-bcf1-43ac-b08e-b83f1fd0a882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:53.283167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcae07f-bcf1-43ac-b08e-b83f1fd0a882 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:53.291972 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 384/752] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:53.292707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcae07f-bcf1-43ac-b08e-b83f1fd0a882 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:53.359522 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6808970-bb1a-4d63-9799-b049ef8bb7f7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:53.392794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:53.405334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97838763-5d1e-44c0-87c3-e6fae609e32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:53.405966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97838763-5d1e-44c0-87c3-e6fae609e32a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:53.406245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97838763-5d1e-44c0-87c3-e6fae609e32a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:53.406514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97838763-5d1e-44c0-87c3-e6fae609e32a None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:53.407315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97838763-5d1e-44c0-87c3-e6fae609e32a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:53.407624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97838763-5d1e-44c0-87c3-e6fae609e32a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:53.409156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97838763-5d1e-44c0-87c3-e6fae609e32a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:53.416608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcae07f-bcf1-43ac-b08e-b83f1fd0a882 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:53.429537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 369/753] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:53.429537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97838763-5d1e-44c0-87c3-e6fae609e32a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:53.465693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538b86d-f5d8-45a2-a856-13ede006bd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:53.466789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538b86d-f5d8-45a2-a856-13ede006bd63 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:53.467304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538b86d-f5d8-45a2-a856-13ede006bd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:53.467814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538b86d-f5d8-45a2-a856-13ede006bd63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:53.475500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538b86d-f5d8-45a2-a856-13ede006bd63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:53.477598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538b86d-f5d8-45a2-a856-13ede006bd63 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-1538b86d-f5d8-45a2-a856-13ede006bd63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:53.490771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:53.491763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:53.493273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:53.493273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:53.493273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:53.493273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:53.493273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:53.493273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:53.493273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:53.493273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:53.496207 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 385/754] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:53.497505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538b86d-f5d8-45a2-a856-13ede006bd63 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:53.537730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97838763-5d1e-44c0-87c3-e6fae609e32a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:53.548482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:53.577183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a37742-fee5-4831-99a4-ada9529d8e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:53.578149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a37742-fee5-4831-99a4-ada9529d8e05 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:53.578568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a37742-fee5-4831-99a4-ada9529d8e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:53.578994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a37742-fee5-4831-99a4-ada9529d8e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:53.590844 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a37742-fee5-4831-99a4-ada9529d8e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:53.592658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a37742-fee5-4831-99a4-ada9529d8e05 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f0a37742-fee5-4831-99a4-ada9529d8e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:53.609418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:53.610419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:53.611406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:53.611406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:53.611406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:53.611406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:53.613304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 370/755] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:53.614225 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a37742-fee5-4831-99a4-ada9529d8e05 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:53.614690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538b86d-f5d8-45a2-a856-13ede006bd63 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:53.656288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78988d-0b4e-441e-9e16-15a9e88ac819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:53.657093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78988d-0b4e-441e-9e16-15a9e88ac819 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:53.657516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78988d-0b4e-441e-9e16-15a9e88ac819 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:53.657899 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78988d-0b4e-441e-9e16-15a9e88ac819 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:53.658956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78988d-0b4e-441e-9e16-15a9e88ac819 None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=8340fafe48af45c8b87a97aaac637bd5) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:53.659541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:53.660680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78988d-0b4e-441e-9e16-15a9e88ac819 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:53.662814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78988d-0b4e-441e-9e16-15a9e88ac819 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:53.679512 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 386/756] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:53 2022] GET /identity/v3/role_assignments?user.id=8340fafe48af45c8b87a97aaac637bd5&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:53.681630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78988d-0b4e-441e-9e16-15a9e88ac819 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:53.727704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a37742-fee5-4831-99a4-ada9529d8e05 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:53.762552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b4a42-4379-4238-8d60-ebb18941d8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:53.763212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b4a42-4379-4238-8d60-ebb18941d8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:53.763498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b4a42-4379-4238-8d60-ebb18941d8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:53.763778 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b4a42-4379-4238-8d60-ebb18941d8c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:53.764526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b4a42-4379-4238-8d60-ebb18941d8c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:53.764835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b4a42-4379-4238-8d60-ebb18941d8c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:53.771149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b4a42-4379-4238-8d60-ebb18941d8c5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:53.780441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 371/757] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:53.792471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b4a42-4379-4238-8d60-ebb18941d8c5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:53.857826 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78988d-0b4e-441e-9e16-15a9e88ac819 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:53.867378 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:53.902228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567dec7e-f23f-4211-b190-96dc56c4a678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:53.903197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dec7e-f23f-4211-b190-96dc56c4a678 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:53.903643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dec7e-f23f-4211-b190-96dc56c4a678 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:53.904080 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dec7e-f23f-4211-b190-96dc56c4a678 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:53.904986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567dec7e-f23f-4211-b190-96dc56c4a678 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:53.905477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567dec7e-f23f-4211-b190-96dc56c4a678 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:53.907135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567dec7e-f23f-4211-b190-96dc56c4a678 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:53.922356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 387/758] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:53.927674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567dec7e-f23f-4211-b190-96dc56c4a678 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:54.003230 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:54.008710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b4a42-4379-4238-8d60-ebb18941d8c5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:54.067394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567dec7e-f23f-4211-b190-96dc56c4a678 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:54.075542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275d693-7468-432e-800b-efaf074d7ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:54.075940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275d693-7468-432e-800b-efaf074d7ebd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:54.076326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275d693-7468-432e-800b-efaf074d7ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:54.076704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275d693-7468-432e-800b-efaf074d7ebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:54.077597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275d693-7468-432e-800b-efaf074d7ebd None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=2ba2655c2eef408498c0298d432263f6) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:54.078262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275d693-7468-432e-800b-efaf074d7ebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:54.079839 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275d693-7468-432e-800b-efaf074d7ebd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:54.101115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 372/759] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:24:53 2022] GET /identity/v3/role_assignments?user.id=2ba2655c2eef408498c0298d432263f6&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:54.105792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fde2ca-c8ef-4f92-aeca-84deb4634d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:54.106742 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fde2ca-c8ef-4f92-aeca-84deb4634d5c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:54.107256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fde2ca-c8ef-4f92-aeca-84deb4634d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:54.107710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fde2ca-c8ef-4f92-aeca-84deb4634d5c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:54.116726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fde2ca-c8ef-4f92-aeca-84deb4634d5c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:54.118999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fde2ca-c8ef-4f92-aeca-84deb4634d5c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a2fde2ca-c8ef-4f92-aeca-84deb4634d5c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:54.132089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:54.133201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:54.134140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:54.134140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:54.134140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:54.134140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:54.134140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:54.134140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:54.134140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:54.134140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:54.134140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:24:54.134140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:54.137366 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 388/760] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Aug 1 07:24:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:54.142819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275d693-7468-432e-800b-efaf074d7ebd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:54.302231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275d693-7468-432e-800b-efaf074d7ebd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:54.341185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf0d33-21fd-4a00-a417-767f3e2416ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:54.341737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf0d33-21fd-4a00-a417-767f3e2416ce None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:54.342022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf0d33-21fd-4a00-a417-767f3e2416ce None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:54.342301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf0d33-21fd-4a00-a417-767f3e2416ce None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:54.343118 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf0d33-21fd-4a00-a417-767f3e2416ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:54.343435 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf0d33-21fd-4a00-a417-767f3e2416ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:54.344855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf0d33-21fd-4a00-a417-767f3e2416ce None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:54.364674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 373/761] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Aug 1 07:24:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:54.370019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fde2ca-c8ef-4f92-aeca-84deb4634d5c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:54.401270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57e513-7905-468d-8c37-b55c13ec8047 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:54.401592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57e513-7905-468d-8c37-b55c13ec8047 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:54.401838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57e513-7905-468d-8c37-b55c13ec8047 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:54.404442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 374/762] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:24:54.408662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e6821-d369-4bee-aad5-8d1bc8d4cb72 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:54.408957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e6821-d369-4bee-aad5-8d1bc8d4cb72 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:54.409240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e6821-d369-4bee-aad5-8d1bc8d4cb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:54.428163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6f2e6821-d369-4bee-aad5-8d1bc8d4cb72 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:54.429310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6f2e6821-d369-4bee-aad5-8d1bc8d4cb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:54.431654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:54.446224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:54.507166 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 375/763] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:24:54.527557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e6821-d369-4bee-aad5-8d1bc8d4cb72 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:54.555382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fde2ca-c8ef-4f92-aeca-84deb4634d5c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:54.591103 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ed272-8801-4d15-8e75-ac46bd2bf533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:54.591503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ed272-8801-4d15-8e75-ac46bd2bf533 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:54.591668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ed272-8801-4d15-8e75-ac46bd2bf533 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:54.594351 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ed272-8801-4d15-8e75-ac46bd2bf533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:54.612345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ed272-8801-4d15-8e75-ac46bd2bf533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:54.613802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ed272-8801-4d15-8e75-ac46bd2bf533 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-c81ed272-8801-4d15-8e75-ac46bd2bf533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:54.632333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:54.633304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:54.635012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:54.635012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:54.635012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:54.635012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:54.635012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 389/764] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:54.639570 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ed272-8801-4d15-8e75-ac46bd2bf533 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:54.693298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e6821-d369-4bee-aad5-8d1bc8d4cb72 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:54.723019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:54.745813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ed272-8801-4d15-8e75-ac46bd2bf533 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:54.765355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e595f9-8dce-4566-a219-c03464a4257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:54.766375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e595f9-8dce-4566-a219-c03464a4257b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:54.766859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e595f9-8dce-4566-a219-c03464a4257b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:54.767478 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e595f9-8dce-4566-a219-c03464a4257b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:54.768312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e595f9-8dce-4566-a219-c03464a4257b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:54.769991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e595f9-8dce-4566-a219-c03464a4257b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:54.784169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cb95a-7664-4eee-8f88-cfec3f2529a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:54.785193 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cb95a-7664-4eee-8f88-cfec3f2529a4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:54.785689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cb95a-7664-4eee-8f88-cfec3f2529a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:54.786178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cb95a-7664-4eee-8f88-cfec3f2529a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:54.787453 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cb95a-7664-4eee-8f88-cfec3f2529a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:54.787978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cb95a-7664-4eee-8f88-cfec3f2529a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:54.790493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cb95a-7664-4eee-8f88-cfec3f2529a4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:54.804672 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 390/765] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-26e595f9-8dce-4566-a219-c03464a4257b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:54.805849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:54.806977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:24:54.807850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:54.810726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cb95a-7664-4eee-8f88-cfec3f2529a4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:54.810953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 376/766] 10.222.0.40 () {56 vars in 1129 bytes} [Mon Aug 1 07:24:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:54.817354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e595f9-8dce-4566-a219-c03464a4257b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:54.884183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:54.926200 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cb95a-7664-4eee-8f88-cfec3f2529a4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:54.970660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93e237-49db-45ff-b902-638b0b58d4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:54.971407 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93e237-49db-45ff-b902-638b0b58d4de None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:54.971696 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93e237-49db-45ff-b902-638b0b58d4de None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:54.971968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93e237-49db-45ff-b902-638b0b58d4de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:54.983323 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93e237-49db-45ff-b902-638b0b58d4de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:54.991586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93e237-49db-45ff-b902-638b0b58d4de None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:54.996815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-7d93e237-49db-45ff-b902-638b0b58d4de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:55.012248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:55.013166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:55.014046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:55.014046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:55.014046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:55.014046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:55.014046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:55.014046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:55.014046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:55.014046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:55.015215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e595f9-8dce-4566-a219-c03464a4257b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:55.024201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 391/767] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:55.029176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93e237-49db-45ff-b902-638b0b58d4de None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:55.073154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79f867-9c2f-4778-9cd7-9fbfd52729cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:55.074165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79f867-9c2f-4778-9cd7-9fbfd52729cf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:55.074644 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79f867-9c2f-4778-9cd7-9fbfd52729cf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:55.075136 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79f867-9c2f-4778-9cd7-9fbfd52729cf None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:55.076019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79f867-9c2f-4778-9cd7-9fbfd52729cf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:55.076933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79f867-9c2f-4778-9cd7-9fbfd52729cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:55.078875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79f867-9c2f-4778-9cd7-9fbfd52729cf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:55.090494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 377/768] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Aug 1 07:24:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:55.092276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79f867-9c2f-4778-9cd7-9fbfd52729cf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:55.225635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93e237-49db-45ff-b902-638b0b58d4de None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:55.261068 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:55.271274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110d136-0955-4e90-a17f-3c2e1531fc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:55.272614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110d136-0955-4e90-a17f-3c2e1531fc46 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:55.273106 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110d136-0955-4e90-a17f-3c2e1531fc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:55.274402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110d136-0955-4e90-a17f-3c2e1531fc46 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:55.275619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110d136-0955-4e90-a17f-3c2e1531fc46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:55.276160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110d136-0955-4e90-a17f-3c2e1531fc46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:55.277798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110d136-0955-4e90-a17f-3c2e1531fc46 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:55.293165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 392/769] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:55.303035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110d136-0955-4e90-a17f-3c2e1531fc46 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:55.313591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79f867-9c2f-4778-9cd7-9fbfd52729cf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:55.348838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79a3ba-733e-4e8b-a70b-994687dc3e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:55.349165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79a3ba-733e-4e8b-a70b-994687dc3e8b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:55.349432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79a3ba-733e-4e8b-a70b-994687dc3e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:55.349711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79a3ba-733e-4e8b-a70b-994687dc3e8b None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:55.350395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79a3ba-733e-4e8b-a70b-994687dc3e8b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:55.350709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79a3ba-733e-4e8b-a70b-994687dc3e8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:55.352507 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79a3ba-733e-4e8b-a70b-994687dc3e8b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:55.357346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 378/770] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Aug 1 07:24:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:55.360646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda081ad-b906-4a01-b642-1bdc8d0086e3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:55.360646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda081ad-b906-4a01-b642-1bdc8d0086e3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:55.360646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda081ad-b906-4a01-b642-1bdc8d0086e3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:55.364353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 379/771] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:55.365563 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762eb93-d54d-4b0d-8ddf-2c3badaf4139 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:55.365853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762eb93-d54d-4b0d-8ddf-2c3badaf4139 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:55.366131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762eb93-d54d-4b0d-8ddf-2c3badaf4139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:55.383893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:55.385300 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1762eb93-d54d-4b0d-8ddf-2c3badaf4139 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:55.386446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1762eb93-d54d-4b0d-8ddf-2c3badaf4139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:55.438983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110d136-0955-4e90-a17f-3c2e1531fc46 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:55.463591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 380/772] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:24:55.464360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762eb93-d54d-4b0d-8ddf-2c3badaf4139 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:55.486720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ae576-7332-4f60-a7d4-c8935f8e08a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:55.487656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ae576-7332-4f60-a7d4-c8935f8e08a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:55.488030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ae576-7332-4f60-a7d4-c8935f8e08a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:55.488403 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ae576-7332-4f60-a7d4-c8935f8e08a7 None admin] PATH_INFO: `/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/abdcd6d09fee45c696449641ea36eae3/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:55.490998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ae576-7332-4f60-a7d4-c8935f8e08a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ddf7d6f2424a85a741d017497b49c9, user_id=abdcd6d09fee45c696449641ea36eae3, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:55.492943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ae576-7332-4f60-a7d4-c8935f8e08a7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:55.511110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 393/773] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:24:55 2022] PUT /identity/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/abdcd6d09fee45c696449641ea36eae3/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:24:55.512203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ae576-7332-4f60-a7d4-c8935f8e08a7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:55.583448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:55.637592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762eb93-d54d-4b0d-8ddf-2c3badaf4139 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:55.708634 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb5c4d-99cd-43ee-89ee-fabdb437d54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:55.709269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb5c4d-99cd-43ee-89ee-fabdb437d54c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:55.709551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb5c4d-99cd-43ee-89ee-fabdb437d54c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:55.709824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb5c4d-99cd-43ee-89ee-fabdb437d54c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:55.710584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb5c4d-99cd-43ee-89ee-fabdb437d54c None admin] RBAC: Adding query filter params (interface=public, service_id=3d21153d628e412a926ef20d1f8e3659, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:55.710895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb5c4d-99cd-43ee-89ee-fabdb437d54c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:55.712342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb5c4d-99cd-43ee-89ee-fabdb437d54c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:55.723100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 381/774] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Aug 1 07:24:55 2022] GET /identity/v3/endpoints?service_id=3d21153d628e412a926ef20d1f8e3659&interface=public®ion_id=RegionOne => generated 194 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:55.728030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb5c4d-99cd-43ee-89ee-fabdb437d54c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:55.731000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ae576-7332-4f60-a7d4-c8935f8e08a7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:55.750408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:55.772487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe267e-3d0c-440a-aeee-801a9eccda12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:55.774302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe267e-3d0c-440a-aeee-801a9eccda12 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:55.774815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe267e-3d0c-440a-aeee-801a9eccda12 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:55.775314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe267e-3d0c-440a-aeee-801a9eccda12 None admin] PATH_INFO: `/v3/services/network` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:55.776168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe267e-3d0c-440a-aeee-801a9eccda12 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:55.777818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe267e-3d0c-440a-aeee-801a9eccda12 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-3dbe267e-3d0c-440a-aeee-801a9eccda12 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:55.790842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:55.792050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:24:55.792866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:55.795628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 394/775] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:55 2022] GET /identity/v3/services/network => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:55.802909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe267e-3d0c-440a-aeee-801a9eccda12 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:55.898299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb5c4d-99cd-43ee-89ee-fabdb437d54c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:55.921632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe267e-3d0c-440a-aeee-801a9eccda12 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:55.972476 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2fbf2-2570-4858-94a2-ab964510dc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:55.973507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2fbf2-2570-4858-94a2-ab964510dc1f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:55.974011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2fbf2-2570-4858-94a2-ab964510dc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:55.974488 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2fbf2-2570-4858-94a2-ab964510dc1f None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:55.975395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2fbf2-2570-4858-94a2-ab964510dc1f None admin] RBAC: Adding query filter params (name=network) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:55.975783 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc05df-5315-49f0-89eb-54e8ba90434f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:55.976162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2fbf2-2570-4858-94a2-ab964510dc1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:55.976428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc05df-5315-49f0-89eb-54e8ba90434f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:55.976823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc05df-5315-49f0-89eb-54e8ba90434f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:55.977202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc05df-5315-49f0-89eb-54e8ba90434f None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:55.977875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc05df-5315-49f0-89eb-54e8ba90434f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:55.978697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2fbf2-2570-4858-94a2-ab964510dc1f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:55.979269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc05df-5315-49f0-89eb-54e8ba90434f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:55.981355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:55.987152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 395/776] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:24:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:55.993386 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2fbf2-2570-4858-94a2-ab964510dc1f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:55.998244 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 382/777] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Aug 1 07:24:55 2022] GET /identity/v3/services => generated 772 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:56.048943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:56.111225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2fbf2-2570-4858-94a2-ab964510dc1f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:56.158844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb5bb0-5450-4fe8-b6f6-3b797fae440a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:56.159577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb5bb0-5450-4fe8-b6f6-3b797fae440a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:56.159872 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb5bb0-5450-4fe8-b6f6-3b797fae440a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:56.160152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb5bb0-5450-4fe8-b6f6-3b797fae440a None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:56.160862 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb5bb0-5450-4fe8-b6f6-3b797fae440a None admin] RBAC: Adding query filter params (type=network) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:56.161174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb5bb0-5450-4fe8-b6f6-3b797fae440a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:56.162605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb5bb0-5450-4fe8-b6f6-3b797fae440a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:56.171168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 396/778] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:24:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:56.177011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc05df-5315-49f0-89eb-54e8ba90434f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:56.323106 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af007baa-0e31-47b8-b183-8b1888a6a675 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:56.323414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af007baa-0e31-47b8-b183-8b1888a6a675 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:56.323626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af007baa-0e31-47b8-b183-8b1888a6a675 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:56.324497 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc05df-5315-49f0-89eb-54e8ba90434f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:56.325230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 397/779] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:56.330343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8faee24-1d93-465b-a7ee-c790a2d69880 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:56.330634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8faee24-1d93-465b-a7ee-c790a2d69880 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:56.330917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8faee24-1d93-465b-a7ee-c790a2d69880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:56.355895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b8faee24-1d93-465b-a7ee-c790a2d69880 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:56.357123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b8faee24-1d93-465b-a7ee-c790a2d69880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:56.365796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599405d7-931a-4701-8853-5c36fb25a6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:56.366422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599405d7-931a-4701-8853-5c36fb25a6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:56.366709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599405d7-931a-4701-8853-5c36fb25a6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:56.366986 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599405d7-931a-4701-8853-5c36fb25a6b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:56.368440 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599405d7-931a-4701-8853-5c36fb25a6b9 None admin] RBAC: Adding query filter params (interface=public, service_id=d116b08e57f44ec2b899515f1b49cda8, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:56.368759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599405d7-931a-4701-8853-5c36fb25a6b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:56.370133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599405d7-931a-4701-8853-5c36fb25a6b9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:56.375580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 383/780] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Aug 1 07:24:56 2022] GET /identity/v3/endpoints?service_id=d116b08e57f44ec2b899515f1b49cda8&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:56.383815 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599405d7-931a-4701-8853-5c36fb25a6b9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:56.468438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:56.469687 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 398/781] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:24:56.516332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8faee24-1d93-465b-a7ee-c790a2d69880 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:56.534718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599405d7-931a-4701-8853-5c36fb25a6b9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:56.587413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:56.589246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8446f7-76d1-4709-a126-42ad5af9d8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:56.590114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8446f7-76d1-4709-a126-42ad5af9d8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:56.590607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8446f7-76d1-4709-a126-42ad5af9d8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:56.591122 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8446f7-76d1-4709-a126-42ad5af9d8a9 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:56.592012 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8446f7-76d1-4709-a126-42ad5af9d8a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:56.593590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8446f7-76d1-4709-a126-42ad5af9d8a9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:56.599735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 384/782] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Aug 1 07:24:56 2022] GET /identity/v3/services => generated 772 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:24:56.682797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8faee24-1d93-465b-a7ee-c790a2d69880 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:56.727255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089262e-fa4d-41f6-91b1-aedcd076fb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:56.727933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089262e-fa4d-41f6-91b1-aedcd076fb36 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:56.728217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089262e-fa4d-41f6-91b1-aedcd076fb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:56.728490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089262e-fa4d-41f6-91b1-aedcd076fb36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:56.742548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089262e-fa4d-41f6-91b1-aedcd076fb36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-d089262e-fa4d-41f6-91b1-aedcd076fb36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:56.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:56.748671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:56.749519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 399/783] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:24:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:56.756637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8446f7-76d1-4709-a126-42ad5af9d8a9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:56.893318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8446f7-76d1-4709-a126-42ad5af9d8a9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:56.913952 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c6501-72f1-4663-8631-9979335d4057 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:56.915724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c6501-72f1-4663-8631-9979335d4057 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:56.915961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c6501-72f1-4663-8631-9979335d4057 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:56.917577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 400/784] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:56.922656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2236e8-a544-44b8-bd49-03e3410e804c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:56.923260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2236e8-a544-44b8-bd49-03e3410e804c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:56.923260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2236e8-a544-44b8-bd49-03e3410e804c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:56.926805 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:56.945376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef1b32-19b3-4b3c-a55b-ec84a532c6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:56.946236 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7f2236e8-a544-44b8-bd49-03e3410e804c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:56.946798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef1b32-19b3-4b3c-a55b-ec84a532c6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:56.947288 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef1b32-19b3-4b3c-a55b-ec84a532c6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:56.947693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7f2236e8-a544-44b8-bd49-03e3410e804c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:56.947975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef1b32-19b3-4b3c-a55b-ec84a532c6e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:56.948962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef1b32-19b3-4b3c-a55b-ec84a532c6e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:56.949448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef1b32-19b3-4b3c-a55b-ec84a532c6e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:56.952201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef1b32-19b3-4b3c-a55b-ec84a532c6e5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:56.964261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 385/785] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:24:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:56.969719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef1b32-19b3-4b3c-a55b-ec84a532c6e5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:57.021693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 401/786] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:24:57.044204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2236e8-a544-44b8-bd49-03e3410e804c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:57.094096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef1b32-19b3-4b3c-a55b-ec84a532c6e5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:57.115508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:57.158699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2236e8-a544-44b8-bd49-03e3410e804c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:57.159762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b1d48-117d-47c2-8bc6-1bc90caca748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:57.160417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1d48-117d-47c2-8bc6-1bc90caca748 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:57.160708 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1d48-117d-47c2-8bc6-1bc90caca748 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:57.160992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1d48-117d-47c2-8bc6-1bc90caca748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:57.179057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b1d48-117d-47c2-8bc6-1bc90caca748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:57.180927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b1d48-117d-47c2-8bc6-1bc90caca748 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:57.194539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c42d1-71d6-4404-8752-777069c25755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:57.195467 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c42d1-71d6-4404-8752-777069c25755 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:57.195841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c42d1-71d6-4404-8752-777069c25755 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:57.196204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c42d1-71d6-4404-8752-777069c25755 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-671b1d48-117d-47c2-8bc6-1bc90caca748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:57.212757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:57.213676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:57.214772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:57.214772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:57.214772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:57.214772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:57.214772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 386/787] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:57.218056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c42d1-71d6-4404-8752-777069c25755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:57.220549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b1d48-117d-47c2-8bc6-1bc90caca748 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a81c42d1-71d6-4404-8752-777069c25755 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:57.224793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:57.225700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:57.226517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:57.226517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:57.226517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:57.226517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:57.226517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:57.226517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:57.226517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:57.226517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:57.226517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:57.226517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:57.226517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:57.226517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:57.234384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 402/788] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:57.243428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c42d1-71d6-4404-8752-777069c25755 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:57.311112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:57.423444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:57.432331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b1d48-117d-47c2-8bc6-1bc90caca748 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:57.442781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c42d1-71d6-4404-8752-777069c25755 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:57.481467 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6747b7-5aef-4d5f-b01e-d9e67fe96702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:57.482097 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6747b7-5aef-4d5f-b01e-d9e67fe96702 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:57.482454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6747b7-5aef-4d5f-b01e-d9e67fe96702 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:57.482748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6747b7-5aef-4d5f-b01e-d9e67fe96702 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:57.483605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6747b7-5aef-4d5f-b01e-d9e67fe96702 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:57.483927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6747b7-5aef-4d5f-b01e-d9e67fe96702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:57.485645 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6747b7-5aef-4d5f-b01e-d9e67fe96702 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:57.494781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 403/789] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:57.500317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6747b7-5aef-4d5f-b01e-d9e67fe96702 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:57.510201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b19949-eb3a-4d44-97d6-31a2c76c2dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:57.510839 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b19949-eb3a-4d44-97d6-31a2c76c2dfd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:57.511153 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b19949-eb3a-4d44-97d6-31a2c76c2dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:57.511440 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b19949-eb3a-4d44-97d6-31a2c76c2dfd None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:57.512149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b19949-eb3a-4d44-97d6-31a2c76c2dfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:57.512465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b19949-eb3a-4d44-97d6-31a2c76c2dfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:57.513846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b19949-eb3a-4d44-97d6-31a2c76c2dfd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:57.557347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 387/790] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:57.564027 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b19949-eb3a-4d44-97d6-31a2c76c2dfd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:57.606100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6747b7-5aef-4d5f-b01e-d9e67fe96702 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:57.644606 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdad42c-6386-4354-b3dc-cad82946a97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:57.645255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdad42c-6386-4354-b3dc-cad82946a97a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:57.645593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdad42c-6386-4354-b3dc-cad82946a97a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:57.645878 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdad42c-6386-4354-b3dc-cad82946a97a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:57.658499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdad42c-6386-4354-b3dc-cad82946a97a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:57.660052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdad42c-6386-4354-b3dc-cad82946a97a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-1fdad42c-6386-4354-b3dc-cad82946a97a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:57.678500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:57.679458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:57.681747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:57.681747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:57.681747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:57.681747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:57.681747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 404/791] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:57.686332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdad42c-6386-4354-b3dc-cad82946a97a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:57.707418 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:57.798401 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdad42c-6386-4354-b3dc-cad82946a97a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:57.835281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2b6ef-795f-4731-b179-92e7cdd6c277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:57.836428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2b6ef-795f-4731-b179-92e7cdd6c277 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:57.836769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2b6ef-795f-4731-b179-92e7cdd6c277 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:57.837046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2b6ef-795f-4731-b179-92e7cdd6c277 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:57.837749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2b6ef-795f-4731-b179-92e7cdd6c277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:57.838058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2b6ef-795f-4731-b179-92e7cdd6c277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:57.839622 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2b6ef-795f-4731-b179-92e7cdd6c277 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:57.852883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 405/792] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:57.858903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2b6ef-795f-4731-b179-92e7cdd6c277 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:57.913047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b19949-eb3a-4d44-97d6-31a2c76c2dfd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:57.915154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:57.928940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:57.970701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cb47e-c532-45bd-90d2-0ca7519d55aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:57.971405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cb47e-c532-45bd-90d2-0ca7519d55aa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:57.971694 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cb47e-c532-45bd-90d2-0ca7519d55aa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:57.971972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cb47e-c532-45bd-90d2-0ca7519d55aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:57.981363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cb47e-c532-45bd-90d2-0ca7519d55aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:57.983059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cb47e-c532-45bd-90d2-0ca7519d55aa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:57.987404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2b6ef-795f-4731-b179-92e7cdd6c277 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-691cb47e-c532-45bd-90d2-0ca7519d55aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:57.997584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:57.998573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:58.000156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:58.000156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:58.000156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:58.000156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:58.000156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:58.000156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:58.000156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:58.000156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:58.000156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:24:58.000156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:58.000156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 388/793] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Aug 1 07:24:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:58.000737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bc3d3-0d6c-4658-808d-253e05e276f7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.000737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bc3d3-0d6c-4658-808d-253e05e276f7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.001075 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bc3d3-0d6c-4658-808d-253e05e276f7 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.006237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 389/794] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:58.006237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0bc3d3-0d6c-4658-808d-253e05e276f7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:58.038470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bbdb3-ef90-4d4d-821d-48afd72b8cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:58.039791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bbdb3-ef90-4d4d-821d-48afd72b8cde None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.040171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bbdb3-ef90-4d4d-821d-48afd72b8cde None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.040607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bbdb3-ef90-4d4d-821d-48afd72b8cde None admin] PATH_INFO: `/v3/users/swift` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.050715 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bbdb3-ef90-4d4d-821d-48afd72b8cde None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:58.052240 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bbdb3-ef90-4d4d-821d-48afd72b8cde None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-755bbdb3-ef90-4d4d-821d-48afd72b8cde None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:58.069598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:58.070666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:58.071655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:58.071655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:58.071655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:58.071655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:58.071655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:58.071655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:58.071655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:58.071655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:58.071655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:24:58.071655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:58.074263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 406/795] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:58.075825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf131a-cd8f-4dd2-9168-e76b3798089c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.076340 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf131a-cd8f-4dd2-9168-e76b3798089c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.076843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf131a-cd8f-4dd2-9168-e76b3798089c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.097579 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bfcf131a-cd8f-4dd2-9168-e76b3798089c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:58.099211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bfcf131a-cd8f-4dd2-9168-e76b3798089c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:58.130029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0bc3d3-0d6c-4658-808d-253e05e276f7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:58.166553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7ece5-6b2c-4edf-aa12-418464745edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:58.167187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7ece5-6b2c-4edf-aa12-418464745edc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.167473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7ece5-6b2c-4edf-aa12-418464745edc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.167749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7ece5-6b2c-4edf-aa12-418464745edc None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.168542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7ece5-6b2c-4edf-aa12-418464745edc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:58.168848 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7ece5-6b2c-4edf-aa12-418464745edc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:58.170371 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7ece5-6b2c-4edf-aa12-418464745edc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:58.185025 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 407/796] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:24:58.186176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf131a-cd8f-4dd2-9168-e76b3798089c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:58.197331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 390/797] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Aug 1 07:24:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:58.197331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7ece5-6b2c-4edf-aa12-418464745edc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:58.217103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:58.269135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:58.298623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7ece5-6b2c-4edf-aa12-418464745edc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:58.321480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf131a-cd8f-4dd2-9168-e76b3798089c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:58.333821 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7336ab-5c06-4ba4-8f07-7567c830ab29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:58.334462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7336ab-5c06-4ba4-8f07-7567c830ab29 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.334748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7336ab-5c06-4ba4-8f07-7567c830ab29 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.335022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7336ab-5c06-4ba4-8f07-7567c830ab29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.345399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7336ab-5c06-4ba4-8f07-7567c830ab29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:58.346887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7336ab-5c06-4ba4-8f07-7567c830ab29 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:58.360750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f47526-4f11-4073-84d4-e576efc64341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:58.361684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f47526-4f11-4073-84d4-e576efc64341 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.362174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f47526-4f11-4073-84d4-e576efc64341 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.362657 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f47526-4f11-4073-84d4-e576efc64341 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0b7336ab-5c06-4ba4-8f07-7567c830ab29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:58.362766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:58.363640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:58.364923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:58.364923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:58.364923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:58.364923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:58.366284 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f47526-4f11-4073-84d4-e576efc64341 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:58.366916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f47526-4f11-4073-84d4-e576efc64341 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:58.368853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 391/798] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:24:58.369232 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f47526-4f11-4073-84d4-e576efc64341 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:58.369672 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7336ab-5c06-4ba4-8f07-7567c830ab29 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:58.389858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 408/799] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Aug 1 07:24:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:58.391248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f47526-4f11-4073-84d4-e576efc64341 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:58.441951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:58.474050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7336ab-5c06-4ba4-8f07-7567c830ab29 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:58.503406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f47526-4f11-4073-84d4-e576efc64341 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:58.509988 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599118b7-c9b2-4fd4-8941-d6e0554ff653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:58.510628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599118b7-c9b2-4fd4-8941-d6e0554ff653 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.510908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599118b7-c9b2-4fd4-8941-d6e0554ff653 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.511219 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599118b7-c9b2-4fd4-8941-d6e0554ff653 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.517909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:58.523642 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599118b7-c9b2-4fd4-8941-d6e0554ff653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-599118b7-c9b2-4fd4-8941-d6e0554ff653 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:24:58.528944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:58.529895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:58.530728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 392/800] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:24:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:58.531977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599118b7-c9b2-4fd4-8941-d6e0554ff653 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:58.546156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053053c-5f54-4475-8fa1-2ad383a1e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:58.546784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053053c-5f54-4475-8fa1-2ad383a1e33a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.547094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053053c-5f54-4475-8fa1-2ad383a1e33a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.547388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053053c-5f54-4475-8fa1-2ad383a1e33a None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.548095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053053c-5f54-4475-8fa1-2ad383a1e33a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:58.548408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053053c-5f54-4475-8fa1-2ad383a1e33a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:58.549780 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053053c-5f54-4475-8fa1-2ad383a1e33a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:58.561769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 409/801] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:58.563084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68317671-7f2f-4716-ae34-f6bc32272348 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.563377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68317671-7f2f-4716-ae34-f6bc32272348 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.563656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68317671-7f2f-4716-ae34-f6bc32272348 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.565030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 410/802] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:58.565631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68317671-7f2f-4716-ae34-f6bc32272348 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:58.643232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599118b7-c9b2-4fd4-8941-d6e0554ff653 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:58.675346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68317671-7f2f-4716-ae34-f6bc32272348 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:58.677102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:58.683653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1b6b5-d6eb-4858-b390-708f1eafcdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:58.684234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1b6b5-d6eb-4858-b390-708f1eafcdb6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.684525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1b6b5-d6eb-4858-b390-708f1eafcdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.684807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1b6b5-d6eb-4858-b390-708f1eafcdb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.696823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1b6b5-d6eb-4858-b390-708f1eafcdb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:58.698208 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1b6b5-d6eb-4858-b390-708f1eafcdb6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:58.713411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de94e0-0c43-4bae-aa4e-48cfc88daf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a8a1b6b5-d6eb-4858-b390-708f1eafcdb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:58.713916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:58.714793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:58.715710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:58.715710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:58.715710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:58.715710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:58.715938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de94e0-0c43-4bae-aa4e-48cfc88daf04 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.715938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de94e0-0c43-4bae-aa4e-48cfc88daf04 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.715938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de94e0-0c43-4bae-aa4e-48cfc88daf04 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.715938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de94e0-0c43-4bae-aa4e-48cfc88daf04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:58.715938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de94e0-0c43-4bae-aa4e-48cfc88daf04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:58.716959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de94e0-0c43-4bae-aa4e-48cfc88daf04 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:58.719791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 393/803] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:58.720546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1b6b5-d6eb-4858-b390-708f1eafcdb6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:58.730394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 411/804] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:24:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:58.730394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae26689-5480-4ef7-ac9a-cc99fd2e0bd1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.730394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae26689-5480-4ef7-ac9a-cc99fd2e0bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.730394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae26689-5480-4ef7-ac9a-cc99fd2e0bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.746678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5ae26689-5480-4ef7-ac9a-cc99fd2e0bd1 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:58.748229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5ae26689-5480-4ef7-ac9a-cc99fd2e0bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:58.777256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:58.835087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 412/805] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:24:58.835087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae26689-5480-4ef7-ac9a-cc99fd2e0bd1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:58.837947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1b6b5-d6eb-4858-b390-708f1eafcdb6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:58.873290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b039a1d-0a6e-47f1-83cd-e6b4866149d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:58.873933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b039a1d-0a6e-47f1-83cd-e6b4866149d0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.874220 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b039a1d-0a6e-47f1-83cd-e6b4866149d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.874500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b039a1d-0a6e-47f1-83cd-e6b4866149d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.882216 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b039a1d-0a6e-47f1-83cd-e6b4866149d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:58.883832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b039a1d-0a6e-47f1-83cd-e6b4866149d0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8b039a1d-0a6e-47f1-83cd-e6b4866149d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:58.895645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:58.896558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:58.897452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:58.897452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:58.897452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:58.897452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:58.897452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:58.897452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:58.897452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:58.897452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:58.897452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 394/806] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:58.898916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b039a1d-0a6e-47f1-83cd-e6b4866149d0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:58.948954 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:58.956107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae26689-5480-4ef7-ac9a-cc99fd2e0bd1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:58.995698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce33f32-ae4d-479f-9a84-63a31744d2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:58.996618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce33f32-ae4d-479f-9a84-63a31744d2bc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:58.996910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce33f32-ae4d-479f-9a84-63a31744d2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:58.997196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce33f32-ae4d-479f-9a84-63a31744d2bc None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:58.997896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce33f32-ae4d-479f-9a84-63a31744d2bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:58.998237 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce33f32-ae4d-479f-9a84-63a31744d2bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:59.000156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce33f32-ae4d-479f-9a84-63a31744d2bc None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:59.011887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b039a1d-0a6e-47f1-83cd-e6b4866149d0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:59.016938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 413/807] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:59.016938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce33f32-ae4d-479f-9a84-63a31744d2bc None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:59.041086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:59.051735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaecb5-27fd-481a-bec7-0b9ee7ece710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:59.052372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaecb5-27fd-481a-bec7-0b9ee7ece710 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.052658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaecb5-27fd-481a-bec7-0b9ee7ece710 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.052936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaecb5-27fd-481a-bec7-0b9ee7ece710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.063652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eaecb5-27fd-481a-bec7-0b9ee7ece710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:59.065085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eaecb5-27fd-481a-bec7-0b9ee7ece710 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-76eaecb5-27fd-481a-bec7-0b9ee7ece710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:24:59.079802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:59.080689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:59.081582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:24:59.081582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:24:59.081582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:24:59.081582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:24:59.081582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 395/808] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:59.081850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaecb5-27fd-481a-bec7-0b9ee7ece710 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:59.119954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce33f32-ae4d-479f-9a84-63a31744d2bc None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:59.154249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaea084-63da-4f89-9f7f-d0ec5a06dfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:59.154891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaea084-63da-4f89-9f7f-d0ec5a06dfd9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.155215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaea084-63da-4f89-9f7f-d0ec5a06dfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.155500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaea084-63da-4f89-9f7f-d0ec5a06dfd9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.156128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaea084-63da-4f89-9f7f-d0ec5a06dfd9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:59.157503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaea084-63da-4f89-9f7f-d0ec5a06dfd9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ceaea084-63da-4f89-9f7f-d0ec5a06dfd9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:59.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:59.167452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:59.168271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 414/809] 10.222.0.40 () {56 vars in 1129 bytes} [Mon Aug 1 07:24:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:59.169108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaea084-63da-4f89-9f7f-d0ec5a06dfd9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:59.177521 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:59.183845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaecb5-27fd-481a-bec7-0b9ee7ece710 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:59.217412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f816d-4c2b-47ed-95cf-32b66d28f00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:59.217950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f816d-4c2b-47ed-95cf-32b66d28f00d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.218231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f816d-4c2b-47ed-95cf-32b66d28f00d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.218506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f816d-4c2b-47ed-95cf-32b66d28f00d None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.219276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f816d-4c2b-47ed-95cf-32b66d28f00d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:59.219581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f816d-4c2b-47ed-95cf-32b66d28f00d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:59.220995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f816d-4c2b-47ed-95cf-32b66d28f00d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:59.232437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 396/810] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:59.233573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bc401-cecc-4dbc-a633-6f3a6e959c3c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.233852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bc401-cecc-4dbc-a633-6f3a6e959c3c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.234127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bc401-cecc-4dbc-a633-6f3a6e959c3c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.235344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 397/811] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:24:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:24:59.236019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bc401-cecc-4dbc-a633-6f3a6e959c3c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:59.263687 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:59.276667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaea084-63da-4f89-9f7f-d0ec5a06dfd9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:59.311376 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610f78d-d644-4699-801c-ac3dbf704ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:59.312011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610f78d-d644-4699-801c-ac3dbf704ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.312298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610f78d-d644-4699-801c-ac3dbf704ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.312576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610f78d-d644-4699-801c-ac3dbf704ed8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.319139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610f78d-d644-4699-801c-ac3dbf704ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:59.321187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610f78d-d644-4699-801c-ac3dbf704ed8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-5610f78d-d644-4699-801c-ac3dbf704ed8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:24:59.332309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:59.333168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:59.339634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:24:59.339634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:24:59.339634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:24:59.339634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:24:59.339634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:24:59.339634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:24:59.339634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:24:59.339634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:59.339634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 415/812] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:59.339634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610f78d-d644-4699-801c-ac3dbf704ed8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:59.340173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bc401-cecc-4dbc-a633-6f3a6e959c3c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:59.374390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b219f-d287-4f4f-8cc6-89683b22f2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:59.374995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b219f-d287-4f4f-8cc6-89683b22f2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.375303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b219f-d287-4f4f-8cc6-89683b22f2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.375584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b219f-d287-4f4f-8cc6-89683b22f2a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.376257 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b219f-d287-4f4f-8cc6-89683b22f2a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:59.376564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b219f-d287-4f4f-8cc6-89683b22f2a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:59.377941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b219f-d287-4f4f-8cc6-89683b22f2a2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:59.389057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 398/813] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:59.389817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b219f-d287-4f4f-8cc6-89683b22f2a2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:59.408526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:59.437261 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610f78d-d644-4699-801c-ac3dbf704ed8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:59.470413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9258f2b-881c-4b76-9033-5e59782f6767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:59.470950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9258f2b-881c-4b76-9033-5e59782f6767 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.471260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9258f2b-881c-4b76-9033-5e59782f6767 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.471542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9258f2b-881c-4b76-9033-5e59782f6767 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.472207 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9258f2b-881c-4b76-9033-5e59782f6767 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:59.472510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9258f2b-881c-4b76-9033-5e59782f6767 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:59.473866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9258f2b-881c-4b76-9033-5e59782f6767 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:59.478750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 416/814] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Aug 1 07:24:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:59.479929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64427c4-c751-4db8-8eea-890680cdf7dd None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.480214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64427c4-c751-4db8-8eea-890680cdf7dd None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.480487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64427c4-c751-4db8-8eea-890680cdf7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.490322 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b219f-d287-4f4f-8cc6-89683b22f2a2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:59.497883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a64427c4-c751-4db8-8eea-890680cdf7dd None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:24:59.499103 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a64427c4-c751-4db8-8eea-890680cdf7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:24:59.506383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:59.524632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702eaa11-70dd-41a2-b655-79980fe42e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:59.525302 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702eaa11-70dd-41a2-b655-79980fe42e4c None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.525593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702eaa11-70dd-41a2-b655-79980fe42e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.525877 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702eaa11-70dd-41a2-b655-79980fe42e4c None admin] PATH_INFO: `/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/8340fafe48af45c8b87a97aaac637bd5/roles/74131e9234dc48a2b55332d15e8c6462` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.540314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702eaa11-70dd-41a2-b655-79980fe42e4c None admin] RBAC: Authorizing `identity:create_grant(project_id=08ddf7d6f2424a85a741d017497b49c9, user_id=8340fafe48af45c8b87a97aaac637bd5, role_id=74131e9234dc48a2b55332d15e8c6462)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:59.542102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702eaa11-70dd-41a2-b655-79980fe42e4c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:59.555358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 399/815] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:24:59 2022] PUT /identity/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/8340fafe48af45c8b87a97aaac637bd5/roles/74131e9234dc48a2b55332d15e8c6462 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:24:59.555948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702eaa11-70dd-41a2-b655-79980fe42e4c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:59.567039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 417/816] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:24:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:24:59.568203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64427c4-c751-4db8-8eea-890680cdf7dd None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:59.642870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:59.684176 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702eaa11-70dd-41a2-b655-79980fe42e4c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:59.701486 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64427c4-c751-4db8-8eea-890680cdf7dd None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:59.731175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f05b9a-37f8-45bc-b626-751afffc45e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:59.731175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f05b9a-37f8-45bc-b626-751afffc45e7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.731175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f05b9a-37f8-45bc-b626-751afffc45e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.731175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f05b9a-37f8-45bc-b626-751afffc45e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.731605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f05b9a-37f8-45bc-b626-751afffc45e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:59.731605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f05b9a-37f8-45bc-b626-751afffc45e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:59.731756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f05b9a-37f8-45bc-b626-751afffc45e7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:59.742174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2050481-cd44-4281-be6d-f84bc3dfc265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:59.743037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2050481-cd44-4281-be6d-f84bc3dfc265 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.743440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2050481-cd44-4281-be6d-f84bc3dfc265 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.743991 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2050481-cd44-4281-be6d-f84bc3dfc265 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.744103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 400/817] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:24:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:59.747109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f05b9a-37f8-45bc-b626-751afffc45e7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:59.754867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2050481-cd44-4281-be6d-f84bc3dfc265 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:59.756355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2050481-cd44-4281-be6d-f84bc3dfc265 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-c2050481-cd44-4281-be6d-f84bc3dfc265 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:24:59.770068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:59.771242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:59.772385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:59.772385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:59.772385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:24:59.772385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:24:59.772385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:24:59.772385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:24:59.772385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:24:59.772385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:24:59.772385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:24:59.772385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:59.772385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 418/818] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Aug 1 07:24:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:59.772385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2050481-cd44-4281-be6d-f84bc3dfc265 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:59.777805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:59.859384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f05b9a-37f8-45bc-b626-751afffc45e7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:59.884345 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:24:59.886873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2050481-cd44-4281-be6d-f84bc3dfc265 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:24:59.894076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a25f7-a78d-47fc-91e2-6167693f5d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:59.894707 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a25f7-a78d-47fc-91e2-6167693f5d42 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.894992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a25f7-a78d-47fc-91e2-6167693f5d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.895710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a25f7-a78d-47fc-91e2-6167693f5d42 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.896400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a25f7-a78d-47fc-91e2-6167693f5d42 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:24:59.896712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a25f7-a78d-47fc-91e2-6167693f5d42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:59.898808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a25f7-a78d-47fc-91e2-6167693f5d42 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:59.908551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 401/819] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Aug 1 07:24:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:24:59.908551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a25f7-a78d-47fc-91e2-6167693f5d42 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:24:59.929025 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30058d97-13b7-49d0-bc41-d7d092dc454c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:24:59.930923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30058d97-13b7-49d0-bc41-d7d092dc454c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:24:59.931271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30058d97-13b7-49d0-bc41-d7d092dc454c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:24:59.931643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30058d97-13b7-49d0-bc41-d7d092dc454c None admin] PATH_INFO: `/v3/services/network` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:24:59.932430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30058d97-13b7-49d0-bc41-d7d092dc454c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:24:59.933800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30058d97-13b7-49d0-bc41-d7d092dc454c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-30058d97-13b7-49d0-bc41-d7d092dc454c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:24:59.942846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:24:59.943979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:24:59.944815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 419/820] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:24:59 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:24:59.945681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30058d97-13b7-49d0-bc41-d7d092dc454c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:00.023029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:00.025827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a25f7-a78d-47fc-91e2-6167693f5d42 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:00.059895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30058d97-13b7-49d0-bc41-d7d092dc454c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:00.061658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3375ec7-2458-4966-b3a4-91011a9d727a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:00.062514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3375ec7-2458-4966-b3a4-91011a9d727a None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:00.062942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3375ec7-2458-4966-b3a4-91011a9d727a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:00.063321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3375ec7-2458-4966-b3a4-91011a9d727a None admin] PATH_INFO: `/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/2ba2655c2eef408498c0298d432263f6/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:00.065265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3375ec7-2458-4966-b3a4-91011a9d727a None admin] RBAC: Authorizing `identity:create_grant(project_id=08ddf7d6f2424a85a741d017497b49c9, user_id=2ba2655c2eef408498c0298d432263f6, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:00.066701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3375ec7-2458-4966-b3a4-91011a9d727a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:00.080742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 402/821] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:24:59 2022] PUT /identity/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/2ba2655c2eef408498c0298d432263f6/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:25:00.081529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3375ec7-2458-4966-b3a4-91011a9d727a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:00.097454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0288ce-cd4a-4934-80f2-d3f146e8e223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:00.098349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0288ce-cd4a-4934-80f2-d3f146e8e223 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:00.098776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0288ce-cd4a-4934-80f2-d3f146e8e223 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:00.099268 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0288ce-cd4a-4934-80f2-d3f146e8e223 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:00.100255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0288ce-cd4a-4934-80f2-d3f146e8e223 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:00.100720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0288ce-cd4a-4934-80f2-d3f146e8e223 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:00.102550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0288ce-cd4a-4934-80f2-d3f146e8e223 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:00.124001 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 420/822] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Aug 1 07:24:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:00.125020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0288ce-cd4a-4934-80f2-d3f146e8e223 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:00.161036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:00.241697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3375ec7-2458-4966-b3a4-91011a9d727a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:00.247116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0288ce-cd4a-4934-80f2-d3f146e8e223 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:00.277515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:00.290103 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54803e1-464a-48af-83c2-7172fcc74197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:00.290725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54803e1-464a-48af-83c2-7172fcc74197 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:00.291007 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54803e1-464a-48af-83c2-7172fcc74197 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:00.291358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54803e1-464a-48af-83c2-7172fcc74197 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:00.292033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54803e1-464a-48af-83c2-7172fcc74197 None admin] RBAC: Adding query filter params (name=network) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:00.292343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54803e1-464a-48af-83c2-7172fcc74197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:00.293460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5fb86-9cd3-4c1f-8805-2fdf67573fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:00.293761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54803e1-464a-48af-83c2-7172fcc74197 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:00.294172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5fb86-9cd3-4c1f-8805-2fdf67573fd9 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:00.294459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5fb86-9cd3-4c1f-8805-2fdf67573fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:00.295015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5fb86-9cd3-4c1f-8805-2fdf67573fd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:00.296436 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5fb86-9cd3-4c1f-8805-2fdf67573fd9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:00.298281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5fb86-9cd3-4c1f-8805-2fdf67573fd9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:00.299720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 421/823] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:25:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:00.300557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54803e1-464a-48af-83c2-7172fcc74197 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:00.324366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 403/824] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:25:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:25:00.325386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5fb86-9cd3-4c1f-8805-2fdf67573fd9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:00.413745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54803e1-464a-48af-83c2-7172fcc74197 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:00.446603 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5fb86-9cd3-4c1f-8805-2fdf67573fd9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:00.452492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:00.479907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a895174-aaab-4909-8314-721f5f60ac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:00.480556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a895174-aaab-4909-8314-721f5f60ac3c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:00.480839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a895174-aaab-4909-8314-721f5f60ac3c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:00.481119 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a895174-aaab-4909-8314-721f5f60ac3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:00.490989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863f530-a743-4e50-85ac-d59f7213bd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:00.492311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863f530-a743-4e50-85ac-d59f7213bd5f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:00.492850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863f530-a743-4e50-85ac-d59f7213bd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:00.493566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863f530-a743-4e50-85ac-d59f7213bd5f None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:00.494407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863f530-a743-4e50-85ac-d59f7213bd5f None admin] RBAC: Adding query filter params (type=network) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:00.494862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863f530-a743-4e50-85ac-d59f7213bd5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:00.496529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863f530-a743-4e50-85ac-d59f7213bd5f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:00.502591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 404/825] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:25:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:00.508937 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863f530-a743-4e50-85ac-d59f7213bd5f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:00.566284 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:00.581862 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a895174-aaab-4909-8314-721f5f60ac3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:00.583589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a895174-aaab-4909-8314-721f5f60ac3c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:00.624735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863f530-a743-4e50-85ac-d59f7213bd5f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:00.664677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61601c-2ca6-4c44-b78f-ba4e920f693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:00.665539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61601c-2ca6-4c44-b78f-ba4e920f693e None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:00.665966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61601c-2ca6-4c44-b78f-ba4e920f693e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:00.666409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61601c-2ca6-4c44-b78f-ba4e920f693e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:00.667343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61601c-2ca6-4c44-b78f-ba4e920f693e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:00.668912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61601c-2ca6-4c44-b78f-ba4e920f693e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:00.697438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 405/826] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:25:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:25:00.706612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 422/827] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:25:00.712011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61601c-2ca6-4c44-b78f-ba4e920f693e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:00.763527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:00.837943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61601c-2ca6-4c44-b78f-ba4e920f693e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:00.876987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311d493-c047-4053-aabb-fdf50cddcf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:00.877643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311d493-c047-4053-aabb-fdf50cddcf39 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:00.877927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311d493-c047-4053-aabb-fdf50cddcf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:00.878202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311d493-c047-4053-aabb-fdf50cddcf39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:00.891686 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311d493-c047-4053-aabb-fdf50cddcf39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:00.893369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311d493-c047-4053-aabb-fdf50cddcf39 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c311d493-c047-4053-aabb-fdf50cddcf39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:00.908730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:00.909706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:00.910589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:00.910589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:00.910589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:00.910589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:00.910589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 406/828] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:00.915797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a895174-aaab-4909-8314-721f5f60ac3c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:01.027453 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:01.047159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a895174-aaab-4909-8314-721f5f60ac3c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:01.094510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b8970-3080-4c15-87d0-fe7d47aff049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:01.095450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b8970-3080-4c15-87d0-fe7d47aff049 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:01.095887 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b8970-3080-4c15-87d0-fe7d47aff049 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:01.096315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b8970-3080-4c15-87d0-fe7d47aff049 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:01.097154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b8970-3080-4c15-87d0-fe7d47aff049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:01.097611 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b8970-3080-4c15-87d0-fe7d47aff049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:01.099157 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b8970-3080-4c15-87d0-fe7d47aff049 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:01.112683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 423/829] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:01.119263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311d493-c047-4053-aabb-fdf50cddcf39 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:01.212527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:01.235802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311d493-c047-4053-aabb-fdf50cddcf39 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:01.277669 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fecaec-5450-4239-afef-b1af1618194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:01.278312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fecaec-5450-4239-afef-b1af1618194e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:01.278593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fecaec-5450-4239-afef-b1af1618194e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:01.278870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fecaec-5450-4239-afef-b1af1618194e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:01.285977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fecaec-5450-4239-afef-b1af1618194e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:01.287479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fecaec-5450-4239-afef-b1af1618194e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f6fecaec-5450-4239-afef-b1af1618194e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:25:01.300726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:01.301667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:01.302648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:25:01.302648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:25:01.302648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:25:01.302648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:25:01.302648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:25:01.302648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:25:01.302648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:01.302648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:01.302648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 407/830] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:01.309621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b8970-3080-4c15-87d0-fe7d47aff049 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:01.422158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b8970-3080-4c15-87d0-fe7d47aff049 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:01.458145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004765e-2a39-46d3-90f3-922fb4b5928b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:01.458763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004765e-2a39-46d3-90f3-922fb4b5928b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:01.459045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004765e-2a39-46d3-90f3-922fb4b5928b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:01.459348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004765e-2a39-46d3-90f3-922fb4b5928b None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:01.460097 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004765e-2a39-46d3-90f3-922fb4b5928b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:01.460406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004765e-2a39-46d3-90f3-922fb4b5928b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:01.461881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004765e-2a39-46d3-90f3-922fb4b5928b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:01.475272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 424/831] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:25:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:01.484144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fecaec-5450-4239-afef-b1af1618194e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:01.594255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fecaec-5450-4239-afef-b1af1618194e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:01.624396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:01.630852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c12cb-1fee-455e-9977-a8a1b0d751c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:01.631713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c12cb-1fee-455e-9977-a8a1b0d751c0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:01.632372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c12cb-1fee-455e-9977-a8a1b0d751c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:01.633094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c12cb-1fee-455e-9977-a8a1b0d751c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:01.634084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c12cb-1fee-455e-9977-a8a1b0d751c0 None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=abdcd6d09fee45c696449641ea36eae3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:01.634735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c12cb-1fee-455e-9977-a8a1b0d751c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:01.636246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c12cb-1fee-455e-9977-a8a1b0d751c0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:01.642859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 408/832] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:25:01 2022] GET /identity/v3/role_assignments?user.id=abdcd6d09fee45c696449641ea36eae3&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:01.762909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595ea15-db74-41c3-a8fa-ac929f749f7c None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:01.763431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595ea15-db74-41c3-a8fa-ac929f749f7c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:01.763431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595ea15-db74-41c3-a8fa-ac929f749f7c None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:01.765032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 425/833] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:01.770589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35df63-999a-4edb-b408-a5d5b7a3010a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:01.770967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35df63-999a-4edb-b408-a5d5b7a3010a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:01.771347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35df63-999a-4edb-b408-a5d5b7a3010a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:01.796102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ec35df63-999a-4edb-b408-a5d5b7a3010a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:01.797454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ec35df63-999a-4edb-b408-a5d5b7a3010a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:01.870651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 409/834] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:01.893355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595ea15-db74-41c3-a8fa-ac929f749f7c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:01.906089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:02.015011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595ea15-db74-41c3-a8fa-ac929f749f7c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:02.051284 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93419dc9-3f8b-4f2f-ab94-4a92a8042628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:02.051933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93419dc9-3f8b-4f2f-ab94-4a92a8042628 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:02.052213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93419dc9-3f8b-4f2f-ab94-4a92a8042628 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:02.052487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93419dc9-3f8b-4f2f-ab94-4a92a8042628 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:02.063365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93419dc9-3f8b-4f2f-ab94-4a92a8042628 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-93419dc9-3f8b-4f2f-ab94-4a92a8042628 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:02.068226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:02.069126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:02.069964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 426/835] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:25:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:02.075820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35df63-999a-4edb-b408-a5d5b7a3010a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:02.183882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35df63-999a-4edb-b408-a5d5b7a3010a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:02.209256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:02.221065 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a079ff-7bc3-4c49-a0cc-36c63a51723b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:02.221748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a079ff-7bc3-4c49-a0cc-36c63a51723b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:02.222384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a079ff-7bc3-4c49-a0cc-36c63a51723b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:02.222670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a079ff-7bc3-4c49-a0cc-36c63a51723b None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:02.223435 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a079ff-7bc3-4c49-a0cc-36c63a51723b None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:02.224182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a079ff-7bc3-4c49-a0cc-36c63a51723b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:02.225561 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a079ff-7bc3-4c49-a0cc-36c63a51723b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:02.234953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 410/836] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:25:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:02.240053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93419dc9-3f8b-4f2f-ab94-4a92a8042628 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:02.312349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:02.349506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93419dc9-3f8b-4f2f-ab94-4a92a8042628 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:02.386281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebffae-e697-4c0a-ba74-dbbaa829f238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:02.386970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebffae-e697-4c0a-ba74-dbbaa829f238 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:02.387283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebffae-e697-4c0a-ba74-dbbaa829f238 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:02.387562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebffae-e697-4c0a-ba74-dbbaa829f238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:02.399129 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebffae-e697-4c0a-ba74-dbbaa829f238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:02.400588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebffae-e697-4c0a-ba74-dbbaa829f238 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-6cebffae-e697-4c0a-ba74-dbbaa829f238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:02.416422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:02.418444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:02.420572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:02.420572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:02.420572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:02.420572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:02.420572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 427/837] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:02.428138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a079ff-7bc3-4c49-a0cc-36c63a51723b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:02.482567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b5ede-8363-46c0-b258-9fbfe2d70ae7 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:02.482894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b5ede-8363-46c0-b258-9fbfe2d70ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:02.483155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b5ede-8363-46c0-b258-9fbfe2d70ae7 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:02.484617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 428/838] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:02.497601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b677cce-9eb9-4d07-ae43-7c2762b260da None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:02.497910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b677cce-9eb9-4d07-ae43-7c2762b260da None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:02.498164 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b677cce-9eb9-4d07-ae43-7c2762b260da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:02.517850 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3b677cce-9eb9-4d07-ae43-7c2762b260da None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:02.519252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3b677cce-9eb9-4d07-ae43-7c2762b260da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:02.541365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a079ff-7bc3-4c49-a0cc-36c63a51723b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:02.575833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3c220-cda2-4c2d-8bf3-be76374bb6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:02.576369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3c220-cda2-4c2d-8bf3-be76374bb6fa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:02.576649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3c220-cda2-4c2d-8bf3-be76374bb6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:02.576931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3c220-cda2-4c2d-8bf3-be76374bb6fa None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:02.577615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e3c220-cda2-4c2d-8bf3-be76374bb6fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:02.577923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e3c220-cda2-4c2d-8bf3-be76374bb6fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:02.579323 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e3c220-cda2-4c2d-8bf3-be76374bb6fa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:02.596742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 411/839] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:02.597682 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3c220-cda2-4c2d-8bf3-be76374bb6fa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:02.600347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 429/840] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:02.624448 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b677cce-9eb9-4d07-ae43-7c2762b260da None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:02.649825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:02.702927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:02.719377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3c220-cda2-4c2d-8bf3-be76374bb6fa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:02.758661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb56a5a-b2c5-44c3-8875-e1e528e64df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:02.759619 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb56a5a-b2c5-44c3-8875-e1e528e64df5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:02.759909 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb56a5a-b2c5-44c3-8875-e1e528e64df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:02.760190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb56a5a-b2c5-44c3-8875-e1e528e64df5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:02.770301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb56a5a-b2c5-44c3-8875-e1e528e64df5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:02.771858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb56a5a-b2c5-44c3-8875-e1e528e64df5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:02.784530 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b677cce-9eb9-4d07-ae43-7c2762b260da None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7bb56a5a-b2c5-44c3-8875-e1e528e64df5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:25:02.791760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:02.793212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:02.794123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:02.794123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:02.794123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:25:02.794123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:25:02.794123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:25:02.794123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:25:02.794123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:25:02.794123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:25:02.794123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:25:02.794123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:02.795232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 412/841] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Aug 1 07:25:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:02.796647 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65e1e3-f566-493c-b0d0-c8ae0c05db89 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:02.797027 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65e1e3-f566-493c-b0d0-c8ae0c05db89 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:02.797231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65e1e3-f566-493c-b0d0-c8ae0c05db89 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:02.799229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 413/842] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:02.799940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65e1e3-f566-493c-b0d0-c8ae0c05db89 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:02.842258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fc852-ace0-410a-ac32-1f80067990b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:02.842835 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fc852-ace0-410a-ac32-1f80067990b9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:02.843180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fc852-ace0-410a-ac32-1f80067990b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:02.843417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fc852-ace0-410a-ac32-1f80067990b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:02.857843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fc852-ace0-410a-ac32-1f80067990b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a62fc852-ace0-410a-ac32-1f80067990b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:02.862893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:02.863888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:02.864726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 430/843] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:25:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:02.865763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2063564-ebca-4614-a296-caf57fd2f607 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:02.865763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2063564-ebca-4614-a296-caf57fd2f607 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:02.865894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2063564-ebca-4614-a296-caf57fd2f607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:02.893097 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f2063564-ebca-4614-a296-caf57fd2f607 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:02.894316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f2063564-ebca-4614-a296-caf57fd2f607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:02.936749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65e1e3-f566-493c-b0d0-c8ae0c05db89 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:02.944286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:02.985463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7c7b3-ebe3-43c6-97c7-0fe1b6efb6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:02.986121 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7c7b3-ebe3-43c6-97c7-0fe1b6efb6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:02.986371 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7c7b3-ebe3-43c6-97c7-0fe1b6efb6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:02.986643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7c7b3-ebe3-43c6-97c7-0fe1b6efb6d3 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:02.987669 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7c7b3-ebe3-43c6-97c7-0fe1b6efb6d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:02.988135 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7c7b3-ebe3-43c6-97c7-0fe1b6efb6d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:02.989464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7c7b3-ebe3-43c6-97c7-0fe1b6efb6d3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:02.994770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 431/844] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:02.995441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2063564-ebca-4614-a296-caf57fd2f607 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:03.022849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 414/845] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Aug 1 07:25:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:03.024506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7c7b3-ebe3-43c6-97c7-0fe1b6efb6d3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:03.045769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:03.116242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2063564-ebca-4614-a296-caf57fd2f607 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:03.156436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b0ff1-8d41-4f22-945c-7c927cca7626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:03.157078 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b0ff1-8d41-4f22-945c-7c927cca7626 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:03.157364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b0ff1-8d41-4f22-945c-7c927cca7626 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:03.158056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7c7b3-ebe3-43c6-97c7-0fe1b6efb6d3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:03.159997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b0ff1-8d41-4f22-945c-7c927cca7626 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:03.162925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b0ff1-8d41-4f22-945c-7c927cca7626 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:03.165470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b0ff1-8d41-4f22-945c-7c927cca7626 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:03.167571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b0ff1-8d41-4f22-945c-7c927cca7626 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:03.185041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 432/846] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:25:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:03.185041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b0ff1-8d41-4f22-945c-7c927cca7626 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:03.206174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae850cac-1612-4721-a380-f171375984dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:03.206819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae850cac-1612-4721-a380-f171375984dd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:03.207129 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae850cac-1612-4721-a380-f171375984dd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:03.207408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae850cac-1612-4721-a380-f171375984dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:03.208418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae850cac-1612-4721-a380-f171375984dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:03.210183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae850cac-1612-4721-a380-f171375984dd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ae850cac-1612-4721-a380-f171375984dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:03.219616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:03.220643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:03.221504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 415/847] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:25:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:03.223606 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae850cac-1612-4721-a380-f171375984dd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:03.256905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:03.316840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b0ff1-8d41-4f22-945c-7c927cca7626 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:03.318834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:03.346614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae850cac-1612-4721-a380-f171375984dd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:03.357015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9092d-9cde-46ec-a9a4-8492bd9850cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:03.357644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9092d-9cde-46ec-a9a4-8492bd9850cc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:03.357929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9092d-9cde-46ec-a9a4-8492bd9850cc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:03.358205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9092d-9cde-46ec-a9a4-8492bd9850cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:03.387574 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf9a50-91fd-4d91-a5f0-eac8c06f3a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:03.388292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf9a50-91fd-4d91-a5f0-eac8c06f3a35 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:03.388581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf9a50-91fd-4d91-a5f0-eac8c06f3a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:03.388862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf9a50-91fd-4d91-a5f0-eac8c06f3a35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:03.400554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf9a50-91fd-4d91-a5f0-eac8c06f3a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:03.402022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf9a50-91fd-4d91-a5f0-eac8c06f3a35 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3acf9a50-91fd-4d91-a5f0-eac8c06f3a35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:03.419413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:03.420390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:03.421326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:03.421326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:03.421326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:03.421326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:03.421326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 416/848] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:03.422123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf9a50-91fd-4d91-a5f0-eac8c06f3a35 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:03.449121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9092d-9cde-46ec-a9a4-8492bd9850cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:03.450586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9092d-9cde-46ec-a9a4-8492bd9850cc None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:03.523156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:03.544527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf9a50-91fd-4d91-a5f0-eac8c06f3a35 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:03.564933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 433/849] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:03 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:25:03.565578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9092d-9cde-46ec-a9a4-8492bd9850cc None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:03.568068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:03.586534 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acb0f7-45a7-4462-ab37-1d9b0c02bd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:03.587190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acb0f7-45a7-4462-ab37-1d9b0c02bd67 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:03.587479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acb0f7-45a7-4462-ab37-1d9b0c02bd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:03.587757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acb0f7-45a7-4462-ab37-1d9b0c02bd67 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:03.588450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acb0f7-45a7-4462-ab37-1d9b0c02bd67 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:03.588743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acb0f7-45a7-4462-ab37-1d9b0c02bd67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:03.590113 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acb0f7-45a7-4462-ab37-1d9b0c02bd67 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:03.599999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 417/850] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:25:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:03.599999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acb0f7-45a7-4462-ab37-1d9b0c02bd67 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:03.677493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9092d-9cde-46ec-a9a4-8492bd9850cc None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:03.711217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acb0f7-45a7-4462-ab37-1d9b0c02bd67 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:03.715694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f15d7-cb7a-441f-b7b2-7727347c8ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:03.716327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f15d7-cb7a-441f-b7b2-7727347c8ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:03.716614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f15d7-cb7a-441f-b7b2-7727347c8ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:03.716987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f15d7-cb7a-441f-b7b2-7727347c8ff9 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:03.717690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f15d7-cb7a-441f-b7b2-7727347c8ff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:03.718002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f15d7-cb7a-441f-b7b2-7727347c8ff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:03.719466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f15d7-cb7a-441f-b7b2-7727347c8ff9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:03.733027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 434/851] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:03.733708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f15d7-cb7a-441f-b7b2-7727347c8ff9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:03.753766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de571f1-2651-4466-902e-4e74f26246c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:03.754409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de571f1-2651-4466-902e-4e74f26246c6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:03.754694 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de571f1-2651-4466-902e-4e74f26246c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:03.754971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de571f1-2651-4466-902e-4e74f26246c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:03.766794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de571f1-2651-4466-902e-4e74f26246c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:03.768285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de571f1-2651-4466-902e-4e74f26246c6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:03.771620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1de571f1-2651-4466-902e-4e74f26246c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:03.785549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:03.786458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:03.787377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:03.787377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:03.787377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:03.787377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:03.787377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 418/852] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:03.788096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de571f1-2651-4466-902e-4e74f26246c6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:03.818303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:03.861015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f15d7-cb7a-441f-b7b2-7727347c8ff9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:03.896034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89dbb2-261d-4499-afac-4edb924bb4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:03.896658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89dbb2-261d-4499-afac-4edb924bb4ec None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:03.896939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89dbb2-261d-4499-afac-4edb924bb4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:03.897214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89dbb2-261d-4499-afac-4edb924bb4ec None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:03.897882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89dbb2-261d-4499-afac-4edb924bb4ec None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:03.898728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89dbb2-261d-4499-afac-4edb924bb4ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:03.900195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89dbb2-261d-4499-afac-4edb924bb4ec None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:03.901597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de571f1-2651-4466-902e-4e74f26246c6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:03.904609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 435/853] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:25:03 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:03.905372 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89dbb2-261d-4499-afac-4edb924bb4ec None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:03.951016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c4388-585b-4c1e-9123-f8ba80765e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:03.952341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c4388-585b-4c1e-9123-f8ba80765e83 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:03.952341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c4388-585b-4c1e-9123-f8ba80765e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:03.952341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c4388-585b-4c1e-9123-f8ba80765e83 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:03.964540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c4388-585b-4c1e-9123-f8ba80765e83 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:03.966019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c4388-585b-4c1e-9123-f8ba80765e83 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8a3c4388-585b-4c1e-9123-f8ba80765e83 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:25:03.980465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:03.981518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:03.982455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:03.982455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:03.982455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:25:03.982455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:25:03.982455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:25:03.982455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:25:03.982455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:25:03.982455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:25:03.982455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 07:25:03.982455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:03.982455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 419/854] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:25:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:03.983048 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db6af1-fed7-49d3-9e67-167b0e1a5691 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:03.984117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db6af1-fed7-49d3-9e67-167b0e1a5691 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:03.984117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db6af1-fed7-49d3-9e67-167b0e1a5691 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:03.989153 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 420/855] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:03.989153 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db6af1-fed7-49d3-9e67-167b0e1a5691 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:04.043245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89dbb2-261d-4499-afac-4edb924bb4ec None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:04.055023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:04.075539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:04.084946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f78c6-1cc4-464d-9b6a-3b482003562d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:04.086118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f78c6-1cc4-464d-9b6a-3b482003562d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:04.086410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f78c6-1cc4-464d-9b6a-3b482003562d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:04.086690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f78c6-1cc4-464d-9b6a-3b482003562d None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:04.087421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f78c6-1cc4-464d-9b6a-3b482003562d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:04.087732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f78c6-1cc4-464d-9b6a-3b482003562d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:04.089105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f78c6-1cc4-464d-9b6a-3b482003562d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:04.102045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 436/856] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:04.103284 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d5300-0927-4ebf-82cc-f4ce7964d6fe None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:04.103561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d5300-0927-4ebf-82cc-f4ce7964d6fe None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:04.103840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d5300-0927-4ebf-82cc-f4ce7964d6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:04.107201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db6af1-fed7-49d3-9e67-167b0e1a5691 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:04.121856 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6f2d5300-0927-4ebf-82cc-f4ce7964d6fe None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:04.123055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6f2d5300-0927-4ebf-82cc-f4ce7964d6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:04.145661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b3a96-9fae-4fd7-bad8-9d8fdeed6af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:04.146278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b3a96-9fae-4fd7-bad8-9d8fdeed6af2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:04.146584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b3a96-9fae-4fd7-bad8-9d8fdeed6af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:04.146857 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b3a96-9fae-4fd7-bad8-9d8fdeed6af2 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:04.147678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b3a96-9fae-4fd7-bad8-9d8fdeed6af2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:04.147984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b3a96-9fae-4fd7-bad8-9d8fdeed6af2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:04.150054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b3a96-9fae-4fd7-bad8-9d8fdeed6af2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:04.173777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 421/857] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Aug 1 07:25:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:04.173777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b3a96-9fae-4fd7-bad8-9d8fdeed6af2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:04.202367 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 437/858] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:04.203999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d5300-0927-4ebf-82cc-f4ce7964d6fe None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:04.279234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b3a96-9fae-4fd7-bad8-9d8fdeed6af2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:04.294308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:04.314905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ea9c7-6c27-477f-83e6-560edc52f37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:04.315582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ea9c7-6c27-477f-83e6-560edc52f37f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:04.315869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ea9c7-6c27-477f-83e6-560edc52f37f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:04.316148 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ea9c7-6c27-477f-83e6-560edc52f37f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:04.319547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d5300-0927-4ebf-82cc-f4ce7964d6fe None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:04.323953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ea9c7-6c27-477f-83e6-560edc52f37f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:04.325398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ea9c7-6c27-477f-83e6-560edc52f37f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:04.337118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c87ea9c7-6c27-477f-83e6-560edc52f37f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:25:04.339531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:04.340496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:04.341415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:25:04.341415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:25:04.341415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:25:04.341415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:25:04.341415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:25:04.341415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:25:04.341415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:04.341415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:04.345494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 422/859] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:04.346388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ea9c7-6c27-477f-83e6-560edc52f37f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:04.357458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92b130-f2e2-49cd-abc9-2f9ef41d16c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:04.358125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92b130-f2e2-49cd-abc9-2f9ef41d16c0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:04.358415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92b130-f2e2-49cd-abc9-2f9ef41d16c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:04.358691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92b130-f2e2-49cd-abc9-2f9ef41d16c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:04.432543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92b130-f2e2-49cd-abc9-2f9ef41d16c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:04.433997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92b130-f2e2-49cd-abc9-2f9ef41d16c0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:04.454546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ea9c7-6c27-477f-83e6-560edc52f37f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:04.489187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fd4d6-31d3-4c01-8ba4-158d9c09cd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:04.490053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fd4d6-31d3-4c01-8ba4-158d9c09cd14 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:04.490516 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fd4d6-31d3-4c01-8ba4-158d9c09cd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:04.490947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fd4d6-31d3-4c01-8ba4-158d9c09cd14 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:04.491777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fd4d6-31d3-4c01-8ba4-158d9c09cd14 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:04.493305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fd4d6-31d3-4c01-8ba4-158d9c09cd14 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f49fd4d6-31d3-4c01-8ba4-158d9c09cd14 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:04.502398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:04.503533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:25:04.504560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:04.506760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 423/860] 10.222.0.40 () {56 vars in 1132 bytes} [Mon Aug 1 07:25:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:04.508546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fd4d6-31d3-4c01-8ba4-158d9c09cd14 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:04.546204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 438/861] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:04 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:25:04.551001 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:04.551333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92b130-f2e2-49cd-abc9-2f9ef41d16c0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:04.572725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:04.616384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fd4d6-31d3-4c01-8ba4-158d9c09cd14 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:04.655615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a076d-d385-42cb-911d-f119d303322f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:04.656230 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a076d-d385-42cb-911d-f119d303322f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:04.656507 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a076d-d385-42cb-911d-f119d303322f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:04.656806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a076d-d385-42cb-911d-f119d303322f None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:04.657546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a076d-d385-42cb-911d-f119d303322f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:04.657852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a076d-d385-42cb-911d-f119d303322f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:04.659295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a076d-d385-42cb-911d-f119d303322f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:04.666363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92b130-f2e2-49cd-abc9-2f9ef41d16c0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:04.673261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 424/862] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:25:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:04.674812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a076d-d385-42cb-911d-f119d303322f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:04.700512 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7de920-158c-483b-aba4-b4e85fc2efed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:04.701125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7de920-158c-483b-aba4-b4e85fc2efed None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:04.701425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7de920-158c-483b-aba4-b4e85fc2efed None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:04.701675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7de920-158c-483b-aba4-b4e85fc2efed None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:04.702370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7de920-158c-483b-aba4-b4e85fc2efed None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:04.702672 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7de920-158c-483b-aba4-b4e85fc2efed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:04.704051 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7de920-158c-483b-aba4-b4e85fc2efed None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:04.714111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 439/863] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Aug 1 07:25:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:04.714111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7de920-158c-483b-aba4-b4e85fc2efed None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:04.781429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:04.785196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a076d-d385-42cb-911d-f119d303322f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:04.800844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:04.822305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ca346-245d-41ee-90b3-cccbfc80f6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:04.822571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7de920-158c-483b-aba4-b4e85fc2efed None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:04.822990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ca346-245d-41ee-90b3-cccbfc80f6fc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:04.823256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ca346-245d-41ee-90b3-cccbfc80f6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:04.823540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ca346-245d-41ee-90b3-cccbfc80f6fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:04.834514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ca346-245d-41ee-90b3-cccbfc80f6fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:04.836463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ca346-245d-41ee-90b3-cccbfc80f6fc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6c5ca346-245d-41ee-90b3-cccbfc80f6fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:04.850704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:04.851994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:04.852890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:04.852890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:04.852890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:04.852890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:04.852890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 425/864] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:04.853172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ca346-245d-41ee-90b3-cccbfc80f6fc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:04.862272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832759b4-9734-47da-92b7-d2d49b5f2243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:04.862891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832759b4-9734-47da-92b7-d2d49b5f2243 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:04.863263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832759b4-9734-47da-92b7-d2d49b5f2243 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:04.863885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832759b4-9734-47da-92b7-d2d49b5f2243 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:04.864681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832759b4-9734-47da-92b7-d2d49b5f2243 None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=8340fafe48af45c8b87a97aaac637bd5) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:04.864989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832759b4-9734-47da-92b7-d2d49b5f2243 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:04.869117 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832759b4-9734-47da-92b7-d2d49b5f2243 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:04.883090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 440/865] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:25:04 2022] GET /identity/v3/role_assignments?user.id=8340fafe48af45c8b87a97aaac637bd5&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:04.883090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832759b4-9734-47da-92b7-d2d49b5f2243 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:04.982841 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ca346-245d-41ee-90b3-cccbfc80f6fc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:05.025205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f614c-a818-4559-abb9-0b6cf360056f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:05.025956 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f614c-a818-4559-abb9-0b6cf360056f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:05.026314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f614c-a818-4559-abb9-0b6cf360056f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:05.026930 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f614c-a818-4559-abb9-0b6cf360056f None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:05.027504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f614c-a818-4559-abb9-0b6cf360056f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:05.027916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f614c-a818-4559-abb9-0b6cf360056f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:05.030564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832759b4-9734-47da-92b7-d2d49b5f2243 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:05.030785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f614c-a818-4559-abb9-0b6cf360056f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:05.047608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 426/866] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Aug 1 07:25:04 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:05.074492 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea65ab-ec59-4ef1-81ed-aed165bb9f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:05.075172 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea65ab-ec59-4ef1-81ed-aed165bb9f9f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:05.075461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea65ab-ec59-4ef1-81ed-aed165bb9f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:05.075742 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea65ab-ec59-4ef1-81ed-aed165bb9f9f None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:05.076460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea65ab-ec59-4ef1-81ed-aed165bb9f9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:05.076770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea65ab-ec59-4ef1-81ed-aed165bb9f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:05.078182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea65ab-ec59-4ef1-81ed-aed165bb9f9f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:05.093100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 441/867] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:05.098959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f614c-a818-4559-abb9-0b6cf360056f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:05.119083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:05.219712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f614c-a818-4559-abb9-0b6cf360056f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:05.256162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae18ed-0645-43a4-bc8f-55be4339b937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:05.256812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae18ed-0645-43a4-bc8f-55be4339b937 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:05.257100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae18ed-0645-43a4-bc8f-55be4339b937 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:05.257435 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae18ed-0645-43a4-bc8f-55be4339b937 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:05.264851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae18ed-0645-43a4-bc8f-55be4339b937 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:05.266296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae18ed-0645-43a4-bc8f-55be4339b937 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-61ae18ed-0645-43a4-bc8f-55be4339b937 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:25:05.278464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:05.279447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:05.280316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:25:05.280316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:25:05.280316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:25:05.280316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:25:05.280316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:25:05.280316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:25:05.280316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:05.280316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:05.280316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 427/868] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:05.285475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea65ab-ec59-4ef1-81ed-aed165bb9f9f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:05.290621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:05.395313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea65ab-ec59-4ef1-81ed-aed165bb9f9f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:05.432704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fd467-5508-4c4e-9a8e-c2eb537e49b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:05.433379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fd467-5508-4c4e-9a8e-c2eb537e49b9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:05.433669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fd467-5508-4c4e-9a8e-c2eb537e49b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:05.433950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fd467-5508-4c4e-9a8e-c2eb537e49b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:05.434706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fd467-5508-4c4e-9a8e-c2eb537e49b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:05.435017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fd467-5508-4c4e-9a8e-c2eb537e49b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:05.436487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fd467-5508-4c4e-9a8e-c2eb537e49b9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:05.450263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 442/869] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:25:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:05.456259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae18ed-0645-43a4-bc8f-55be4339b937 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:05.544042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:05.572644 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae18ed-0645-43a4-bc8f-55be4339b937 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:05.609568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303a34e-19f1-40f7-bf2b-e4743dfa269f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:05.610460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303a34e-19f1-40f7-bf2b-e4743dfa269f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:05.610907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303a34e-19f1-40f7-bf2b-e4743dfa269f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:05.611358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303a34e-19f1-40f7-bf2b-e4743dfa269f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:05.612307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303a34e-19f1-40f7-bf2b-e4743dfa269f None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=2ba2655c2eef408498c0298d432263f6) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:05.612770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303a34e-19f1-40f7-bf2b-e4743dfa269f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:05.614428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303a34e-19f1-40f7-bf2b-e4743dfa269f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:05.622262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 428/870] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:25:05 2022] GET /identity/v3/role_assignments?user.id=2ba2655c2eef408498c0298d432263f6&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:05.955859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9981f99-cf3b-4f04-9009-f93fde7a2434 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:05.956375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9981f99-cf3b-4f04-9009-f93fde7a2434 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:05.956461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9981f99-cf3b-4f04-9009-f93fde7a2434 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:05.957987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 443/871] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:05.963594 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf422d-cb9d-443f-9535-5e151f33206b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:05.963894 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf422d-cb9d-443f-9535-5e151f33206b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:05.964179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf422d-cb9d-443f-9535-5e151f33206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:05.982978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-13bf422d-cb9d-443f-9535-5e151f33206b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:05.984200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13bf422d-cb9d-443f-9535-5e151f33206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:06.053960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 429/872] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:06.076735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9981f99-cf3b-4f04-9009-f93fde7a2434 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:06.146702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:06.192916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9981f99-cf3b-4f04-9009-f93fde7a2434 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:06.231176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343301d3-00ee-4bfa-b434-5d35f2767be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:06.231744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343301d3-00ee-4bfa-b434-5d35f2767be5 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:06.232311 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343301d3-00ee-4bfa-b434-5d35f2767be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:06.232634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343301d3-00ee-4bfa-b434-5d35f2767be5 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:06.233334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343301d3-00ee-4bfa-b434-5d35f2767be5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:06.234751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343301d3-00ee-4bfa-b434-5d35f2767be5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:06.253339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 444/873] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:25:06 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:25:07.069853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f2018-9c02-4c60-a961-f3a809109c25 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:07.070417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f2018-9c02-4c60-a961-f3a809109c25 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:07.070556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f2018-9c02-4c60-a961-f3a809109c25 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:07.072127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 430/874] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:07.077147 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730b227-a51e-4c41-bffa-44573ef99362 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:07.077536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730b227-a51e-4c41-bffa-44573ef99362 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:07.077702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730b227-a51e-4c41-bffa-44573ef99362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:07.096567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7730b227-a51e-4c41-bffa-44573ef99362 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:07.097748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7730b227-a51e-4c41-bffa-44573ef99362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:07.153524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028d6fb-bbe7-4837-8b2e-d33b1f95a9fb None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:07.153833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028d6fb-bbe7-4837-8b2e-d33b1f95a9fb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:07.154070 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028d6fb-bbe7-4837-8b2e-d33b1f95a9fb None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:07.155659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 431/875] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:07.166657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1595d50-740a-4349-a018-a7d6fa9a082c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:07.167256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1595d50-740a-4349-a018-a7d6fa9a082c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:07.167736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1595d50-740a-4349-a018-a7d6fa9a082c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:07.177640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 445/876] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:07 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:07.186374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e1595d50-740a-4349-a018-a7d6fa9a082c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:07.187819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e1595d50-740a-4349-a018-a7d6fa9a082c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:07.201556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730b227-a51e-4c41-bffa-44573ef99362 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:07.255798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:07.267229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:07.277215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 432/877] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:07 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:07.300185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1595d50-740a-4349-a018-a7d6fa9a082c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:07.327865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730b227-a51e-4c41-bffa-44573ef99362 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:07.375230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0a857-7153-454d-ad9d-e3b728d0b344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:07.377234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0a857-7153-454d-ad9d-e3b728d0b344 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:07.377704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0a857-7153-454d-ad9d-e3b728d0b344 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:07.378155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0a857-7153-454d-ad9d-e3b728d0b344 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:07.394023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0a857-7153-454d-ad9d-e3b728d0b344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ebd0a857-7153-454d-ad9d-e3b728d0b344 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:07.401414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:07.402429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:07.403309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:07.403309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:07.403309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:07.403309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:07.403309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:07.403309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:07.403309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:07.403309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:07.403309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:07.403309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:07.403309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:25:07.403309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:07.407427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 446/878] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Aug 1 07:25:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:07.411310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0a857-7153-454d-ad9d-e3b728d0b344 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:07.418985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1595d50-740a-4349-a018-a7d6fa9a082c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:07.455458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32528149-5e51-45d1-b5e8-3865a085c22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:07.456081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32528149-5e51-45d1-b5e8-3865a085c22d None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:07.456361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32528149-5e51-45d1-b5e8-3865a085c22d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:07.456637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32528149-5e51-45d1-b5e8-3865a085c22d None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:07.457322 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32528149-5e51-45d1-b5e8-3865a085c22d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:07.458680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32528149-5e51-45d1-b5e8-3865a085c22d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:07.487095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 433/879] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:25:07 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:25:07.547848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0a857-7153-454d-ad9d-e3b728d0b344 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:07.560070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:07.599427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdad2c-5ae2-4ae0-a88f-53b8863eae7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:07.600456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdad2c-5ae2-4ae0-a88f-53b8863eae7f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:07.601019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdad2c-5ae2-4ae0-a88f-53b8863eae7f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:07.602023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdad2c-5ae2-4ae0-a88f-53b8863eae7f None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:07.602944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdad2c-5ae2-4ae0-a88f-53b8863eae7f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:07.603503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdad2c-5ae2-4ae0-a88f-53b8863eae7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:07.605113 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdad2c-5ae2-4ae0-a88f-53b8863eae7f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:07.615178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 447/880] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Aug 1 07:25:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:07.623988 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32528149-5e51-45d1-b5e8-3865a085c22d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:07.686728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:07.731510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32528149-5e51-45d1-b5e8-3865a085c22d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:07.766494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7394075-b69d-4f23-b512-68a3ae0a906c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:07.767156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7394075-b69d-4f23-b512-68a3ae0a906c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:07.767437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7394075-b69d-4f23-b512-68a3ae0a906c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:07.767712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7394075-b69d-4f23-b512-68a3ae0a906c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:07.779190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7394075-b69d-4f23-b512-68a3ae0a906c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:07.780634 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7394075-b69d-4f23-b512-68a3ae0a906c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f7394075-b69d-4f23-b512-68a3ae0a906c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:07.795657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:07.796591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:07.797946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:07.797946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:07.797946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:07.797946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:07.797946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 434/881] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:07.802699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdad2c-5ae2-4ae0-a88f-53b8863eae7f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:07.821678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23be4da-ea23-4b46-90f6-65015f60a695 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:07.821678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23be4da-ea23-4b46-90f6-65015f60a695 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:07.821954 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23be4da-ea23-4b46-90f6-65015f60a695 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:07.823572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 435/882] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:25:07.828851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87421c33-f87b-4564-88dd-03533acb7718 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:07.829133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87421c33-f87b-4564-88dd-03533acb7718 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:07.829406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87421c33-f87b-4564-88dd-03533acb7718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:07.848876 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-87421c33-f87b-4564-88dd-03533acb7718 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:07.850067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-87421c33-f87b-4564-88dd-03533acb7718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:07.931271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdad2c-5ae2-4ae0-a88f-53b8863eae7f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:07.931485 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 436/883] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:07.950106 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87421c33-f87b-4564-88dd-03533acb7718 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:07.969304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c755db1-c930-4587-b528-bdd0b0d56cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:07.970328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c755db1-c930-4587-b528-bdd0b0d56cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:07.971220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c755db1-c930-4587-b528-bdd0b0d56cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:07.971708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c755db1-c930-4587-b528-bdd0b0d56cb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:07.972846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c755db1-c930-4587-b528-bdd0b0d56cb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:07.973382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c755db1-c930-4587-b528-bdd0b0d56cb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:07.975377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c755db1-c930-4587-b528-bdd0b0d56cb8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:07.992115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 448/884] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:07.998849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c755db1-c930-4587-b528-bdd0b0d56cb8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:08.006941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:08.044178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:08.089322 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87421c33-f87b-4564-88dd-03533acb7718 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:08.134073 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ef154-cf08-4ecc-bae5-47a913144da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:08.135360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ef154-cf08-4ecc-bae5-47a913144da3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:08.135859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ef154-cf08-4ecc-bae5-47a913144da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:08.136346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ef154-cf08-4ecc-bae5-47a913144da3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:08.137169 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ef154-cf08-4ecc-bae5-47a913144da3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:08.137434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c755db1-c930-4587-b528-bdd0b0d56cb8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:08.139305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ef154-cf08-4ecc-bae5-47a913144da3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-724ef154-cf08-4ecc-bae5-47a913144da3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:08.148872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:08.149853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:08.150665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:08.153415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 437/885] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Aug 1 07:25:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:08.163177 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ef154-cf08-4ecc-bae5-47a913144da3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:08.173684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb27bf-168d-4d5f-9fcc-634d6e7e6811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:08.174329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb27bf-168d-4d5f-9fcc-634d6e7e6811 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:08.174660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb27bf-168d-4d5f-9fcc-634d6e7e6811 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:08.174939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb27bf-168d-4d5f-9fcc-634d6e7e6811 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:08.183148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb27bf-168d-4d5f-9fcc-634d6e7e6811 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:08.184652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb27bf-168d-4d5f-9fcc-634d6e7e6811 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-7ccb27bf-168d-4d5f-9fcc-634d6e7e6811 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:25:08.196984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:08.198030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:08.198957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:08.198957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:08.198957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:25:08.198957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:25:08.198957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:25:08.198957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:25:08.198957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:25:08.198957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:25:08.198957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:25:08.198957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:08.202915 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 449/886] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Aug 1 07:25:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:08.204227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb27bf-168d-4d5f-9fcc-634d6e7e6811 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:08.282954 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ef154-cf08-4ecc-bae5-47a913144da3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:08.299050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:08.299634 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:08.314483 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb27bf-168d-4d5f-9fcc-634d6e7e6811 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:08.321345 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d49256-26f7-4efe-b4ed-e95a8d0cac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:08.321967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d49256-26f7-4efe-b4ed-e95a8d0cac56 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:08.322247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d49256-26f7-4efe-b4ed-e95a8d0cac56 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:08.322518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d49256-26f7-4efe-b4ed-e95a8d0cac56 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:08.323233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d49256-26f7-4efe-b4ed-e95a8d0cac56 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:08.323536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d49256-26f7-4efe-b4ed-e95a8d0cac56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:08.324896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d49256-26f7-4efe-b4ed-e95a8d0cac56 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:08.329729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 438/887] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Aug 1 07:25:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:08.330877 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc607790-d684-488d-8be8-c70d3c72834f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:08.331293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc607790-d684-488d-8be8-c70d3c72834f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:08.331490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc607790-d684-488d-8be8-c70d3c72834f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:08.332897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 439/888] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:08.335918 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc607790-d684-488d-8be8-c70d3c72834f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:08.353547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab3b3d-29b9-478e-a72c-bf4eca356cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:08.354093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab3b3d-29b9-478e-a72c-bf4eca356cdb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:08.354375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab3b3d-29b9-478e-a72c-bf4eca356cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:08.354658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab3b3d-29b9-478e-a72c-bf4eca356cdb None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:08.355468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab3b3d-29b9-478e-a72c-bf4eca356cdb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:08.355776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab3b3d-29b9-478e-a72c-bf4eca356cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:08.357201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab3b3d-29b9-478e-a72c-bf4eca356cdb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:08.379118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 450/889] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Aug 1 07:25:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:08.379118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16be5e-07fe-4408-88b0-37bc6fc0030e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:08.379118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16be5e-07fe-4408-88b0-37bc6fc0030e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:08.379118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16be5e-07fe-4408-88b0-37bc6fc0030e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:08.395312 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3c16be5e-07fe-4408-88b0-37bc6fc0030e None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:08.396490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3c16be5e-07fe-4408-88b0-37bc6fc0030e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:08.452566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc607790-d684-488d-8be8-c70d3c72834f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:08.471018 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 451/890] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:08.471917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16be5e-07fe-4408-88b0-37bc6fc0030e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:08.494215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca06fd-06d5-4b1d-a459-0411937924b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:08.494832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca06fd-06d5-4b1d-a459-0411937924b6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:08.495138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca06fd-06d5-4b1d-a459-0411937924b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:08.495413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca06fd-06d5-4b1d-a459-0411937924b6 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:08.496087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca06fd-06d5-4b1d-a459-0411937924b6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:08.496389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca06fd-06d5-4b1d-a459-0411937924b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:08.497810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca06fd-06d5-4b1d-a459-0411937924b6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:08.502265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 440/891] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Aug 1 07:25:08 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:08.503023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca06fd-06d5-4b1d-a459-0411937924b6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:08.552573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:08.577347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:08.607056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16be5e-07fe-4408-88b0-37bc6fc0030e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:08.649584 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2b953-40f9-45a4-8db0-cf9942f33356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:08.650223 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2b953-40f9-45a4-8db0-cf9942f33356 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:08.650503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2b953-40f9-45a4-8db0-cf9942f33356 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:08.650774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2b953-40f9-45a4-8db0-cf9942f33356 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:08.659922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca06fd-06d5-4b1d-a459-0411937924b6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:08.706379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667965d-99c9-432c-9115-27ca30ccfaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:08.707455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667965d-99c9-432c-9115-27ca30ccfaaa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:08.708361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667965d-99c9-432c-9115-27ca30ccfaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:08.709366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667965d-99c9-432c-9115-27ca30ccfaaa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:08.710738 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667965d-99c9-432c-9115-27ca30ccfaaa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:08.713602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667965d-99c9-432c-9115-27ca30ccfaaa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0667965d-99c9-432c-9115-27ca30ccfaaa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:08.728734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:08.729785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:25:08.730632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:08.732626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 441/892] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:25:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:08.738571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667965d-99c9-432c-9115-27ca30ccfaaa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:08.741395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f2b953-40f9-45a4-8db0-cf9942f33356 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:08.743542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f2b953-40f9-45a4-8db0-cf9942f33356 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:08.844163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:08.873997 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:08.877537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 452/893] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:08 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:25:08.883159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2b953-40f9-45a4-8db0-cf9942f33356 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:08.885883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667965d-99c9-432c-9115-27ca30ccfaaa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:08.934276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6534b86-28e3-4d06-a53d-63ae681adb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:08.935256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6534b86-28e3-4d06-a53d-63ae681adb1b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:08.935794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6534b86-28e3-4d06-a53d-63ae681adb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:08.936079 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6534b86-28e3-4d06-a53d-63ae681adb1b None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:08.936767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6534b86-28e3-4d06-a53d-63ae681adb1b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:08.937074 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6534b86-28e3-4d06-a53d-63ae681adb1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:08.939169 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6534b86-28e3-4d06-a53d-63ae681adb1b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:08.947115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 442/894] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:25:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:08.952746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6534b86-28e3-4d06-a53d-63ae681adb1b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:09.002196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2b953-40f9-45a4-8db0-cf9942f33356 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:09.039095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653ca54-53e1-486f-a568-c18850179150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:09.040199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653ca54-53e1-486f-a568-c18850179150 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:09.040673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653ca54-53e1-486f-a568-c18850179150 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:09.041015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653ca54-53e1-486f-a568-c18850179150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:09.051886 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653ca54-53e1-486f-a568-c18850179150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:09.053489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653ca54-53e1-486f-a568-c18850179150 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-8653ca54-53e1-486f-a568-c18850179150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:09.068818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:09.070114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:09.071182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:09.071182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:09.071182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:09.071182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:09.072371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 453/895] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:09.077929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653ca54-53e1-486f-a568-c18850179150 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:09.088741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6534b86-28e3-4d06-a53d-63ae681adb1b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:09.110022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:09.135957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cc9db-1827-4186-8b7a-d8b2cde0c6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:09.136537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cc9db-1827-4186-8b7a-d8b2cde0c6fa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:09.136951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cc9db-1827-4186-8b7a-d8b2cde0c6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:09.137307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cc9db-1827-4186-8b7a-d8b2cde0c6fa None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:09.138062 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cc9db-1827-4186-8b7a-d8b2cde0c6fa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:09.138441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cc9db-1827-4186-8b7a-d8b2cde0c6fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:09.140019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cc9db-1827-4186-8b7a-d8b2cde0c6fa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:09.150267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 443/896] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:25:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:09.151344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cc9db-1827-4186-8b7a-d8b2cde0c6fa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:09.178213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:09.209983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653ca54-53e1-486f-a568-c18850179150 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:09.255602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3476874-3e8c-4dbd-8720-a03a1ef3b1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:09.256938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3476874-3e8c-4dbd-8720-a03a1ef3b1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:09.257543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3476874-3e8c-4dbd-8720-a03a1ef3b1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:09.258101 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3476874-3e8c-4dbd-8720-a03a1ef3b1c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:09.258970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3476874-3e8c-4dbd-8720-a03a1ef3b1c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:09.259736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3476874-3e8c-4dbd-8720-a03a1ef3b1c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:09.259857 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cc9db-1827-4186-8b7a-d8b2cde0c6fa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:09.261572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3476874-3e8c-4dbd-8720-a03a1ef3b1c1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:09.278322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 454/897] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:09.283608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3476874-3e8c-4dbd-8720-a03a1ef3b1c1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:09.300922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86170581-d390-49b7-85a6-297822d2ecd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:09.301557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86170581-d390-49b7-85a6-297822d2ecd0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:09.301844 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86170581-d390-49b7-85a6-297822d2ecd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:09.302126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86170581-d390-49b7-85a6-297822d2ecd0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:09.302945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86170581-d390-49b7-85a6-297822d2ecd0 None admin] RBAC: Adding query filter params (interface=public, service_id=12ac2867709f41efb6cc22161528e465, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:09.303290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86170581-d390-49b7-85a6-297822d2ecd0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:09.305200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86170581-d390-49b7-85a6-297822d2ecd0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:09.310582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 444/898] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Aug 1 07:25:09 2022] GET /identity/v3/endpoints?service_id=12ac2867709f41efb6cc22161528e465&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:09.315730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86170581-d390-49b7-85a6-297822d2ecd0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:09.403144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:09.417876 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:09.423273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3476874-3e8c-4dbd-8720-a03a1ef3b1c1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:09.431811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86170581-d390-49b7-85a6-297822d2ecd0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:09.466769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9ca8d-e1ed-4955-828f-46aecc97936d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:09.467789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9ca8d-e1ed-4955-828f-46aecc97936d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:09.468288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9ca8d-e1ed-4955-828f-46aecc97936d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:09.468764 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9ca8d-e1ed-4955-828f-46aecc97936d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:09.469035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7572aad-c914-4553-8cf5-d6b8206fe2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:09.469658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7572aad-c914-4553-8cf5-d6b8206fe2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:09.470110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7572aad-c914-4553-8cf5-d6b8206fe2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:09.470401 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7572aad-c914-4553-8cf5-d6b8206fe2d6 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:09.471057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7572aad-c914-4553-8cf5-d6b8206fe2d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:09.472809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7572aad-c914-4553-8cf5-d6b8206fe2d6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:09.477973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9ca8d-e1ed-4955-828f-46aecc97936d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:09.479940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9ca8d-e1ed-4955-828f-46aecc97936d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:09.483253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 445/899] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Aug 1 07:25:09 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-e0e9ca8d-e1ed-4955-828f-46aecc97936d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:25:09.499984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:09.500941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:09.501813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:25:09.501813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:25:09.501813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:25:09.501813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:25:09.501813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:25:09.501813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:25:09.501813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:09.501813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:09.504544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 455/900] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:09.511943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7572aad-c914-4553-8cf5-d6b8206fe2d6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:09.659484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80470afa-655f-4172-bcf1-83df8061fcb0 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:09.660617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80470afa-655f-4172-bcf1-83df8061fcb0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:09.662016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80470afa-655f-4172-bcf1-83df8061fcb0 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:09.666733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 456/901] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:09.671443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e49b3-9239-4214-8d4a-70821c21a1bb None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:09.672062 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e49b3-9239-4214-8d4a-70821c21a1bb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:09.672566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e49b3-9239-4214-8d4a-70821c21a1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:09.687393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7572aad-c914-4553-8cf5-d6b8206fe2d6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:09.697976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9f1e49b3-9239-4214-8d4a-70821c21a1bb None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:09.699901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9f1e49b3-9239-4214-8d4a-70821c21a1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:09.728582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dd2ed-432d-482e-a13b-da715aef0d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:09.729361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dd2ed-432d-482e-a13b-da715aef0d27 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:09.729645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dd2ed-432d-482e-a13b-da715aef0d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:09.730256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dd2ed-432d-482e-a13b-da715aef0d27 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:09.731009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dd2ed-432d-482e-a13b-da715aef0d27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:09.731367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dd2ed-432d-482e-a13b-da715aef0d27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:09.732792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dd2ed-432d-482e-a13b-da715aef0d27 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:09.746867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 446/902] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:25:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:09.751891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dd2ed-432d-482e-a13b-da715aef0d27 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:09.770556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:09.785726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 457/903] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:09.809652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e49b3-9239-4214-8d4a-70821c21a1bb None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:09.865300 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dd2ed-432d-482e-a13b-da715aef0d27 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:09.903338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d3006-f509-4497-b50b-0ec3431a6d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:09.904063 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d3006-f509-4497-b50b-0ec3431a6d2c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:09.904417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d3006-f509-4497-b50b-0ec3431a6d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:09.904782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d3006-f509-4497-b50b-0ec3431a6d2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:09.905648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d3006-f509-4497-b50b-0ec3431a6d2c None admin] RBAC: Adding query filter params (role.id=d1e0bfc9cbb14751b3daabc5e2de9813, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=8340fafe48af45c8b87a97aaac637bd5) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:09.906120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d3006-f509-4497-b50b-0ec3431a6d2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:09.907386 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:09.907717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d3006-f509-4497-b50b-0ec3431a6d2c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:09.915203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 447/904] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:25:09 2022] GET /identity/v3/role_assignments?user.id=8340fafe48af45c8b87a97aaac637bd5&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=d1e0bfc9cbb14751b3daabc5e2de9813 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:09.950838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e49b3-9239-4214-8d4a-70821c21a1bb None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:09.990857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d2bbf-83c0-4145-8418-ab11bfe875a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:09.992225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d2bbf-83c0-4145-8418-ab11bfe875a8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:09.992740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d2bbf-83c0-4145-8418-ab11bfe875a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:09.993177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d2bbf-83c0-4145-8418-ab11bfe875a8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:09.993831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d2bbf-83c0-4145-8418-ab11bfe875a8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:09.995271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d2bbf-83c0-4145-8418-ab11bfe875a8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a90d2bbf-83c0-4145-8418-ab11bfe875a8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:10.005280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:10.006301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:10.007769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 458/905] 10.222.0.40 () {56 vars in 1132 bytes} [Mon Aug 1 07:25:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:10.014121 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d3006-f509-4497-b50b-0ec3431a6d2c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:10.101431 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:10.130185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d3006-f509-4497-b50b-0ec3431a6d2c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:10.168025 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935dda9d-8542-42ae-a186-59f9c103d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:10.168748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935dda9d-8542-42ae-a186-59f9c103d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:10.168929 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935dda9d-8542-42ae-a186-59f9c103d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:10.169212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935dda9d-8542-42ae-a186-59f9c103d9c4 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:10.169893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935dda9d-8542-42ae-a186-59f9c103d9c4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:10.170203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935dda9d-8542-42ae-a186-59f9c103d9c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:10.171620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935dda9d-8542-42ae-a186-59f9c103d9c4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:10.177084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 448/906] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Aug 1 07:25:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:10.182702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d2bbf-83c0-4145-8418-ab11bfe875a8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:10.290341 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d2bbf-83c0-4145-8418-ab11bfe875a8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:10.326894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a40d2-09aa-4f65-b8b3-0de2523361af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:10.327557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a40d2-09aa-4f65-b8b3-0de2523361af None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:10.327843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a40d2-09aa-4f65-b8b3-0de2523361af None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:10.328122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a40d2-09aa-4f65-b8b3-0de2523361af None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:10.328799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a40d2-09aa-4f65-b8b3-0de2523361af None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:10.329107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a40d2-09aa-4f65-b8b3-0de2523361af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:10.330487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a40d2-09aa-4f65-b8b3-0de2523361af None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:10.336614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 459/907] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Aug 1 07:25:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:10.342369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935dda9d-8542-42ae-a186-59f9c103d9c4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:10.449020 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935dda9d-8542-42ae-a186-59f9c103d9c4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:10.484489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c491f-c99f-49ba-a6cd-bbd3a0fb389d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:10.485114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c491f-c99f-49ba-a6cd-bbd3a0fb389d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:10.485399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c491f-c99f-49ba-a6cd-bbd3a0fb389d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:10.485677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c491f-c99f-49ba-a6cd-bbd3a0fb389d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:10.486433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c491f-c99f-49ba-a6cd-bbd3a0fb389d None admin] RBAC: Adding query filter params (interface=public, service_id=36b508aa7fef4776be17bfc98973b064, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:10.486745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c491f-c99f-49ba-a6cd-bbd3a0fb389d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:10.488155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c491f-c99f-49ba-a6cd-bbd3a0fb389d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:10.494237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 449/908] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Aug 1 07:25:10 2022] GET /identity/v3/endpoints?service_id=36b508aa7fef4776be17bfc98973b064&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:10.499734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a40d2-09aa-4f65-b8b3-0de2523361af None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:10.510507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:10.563631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef067e-d863-4a62-b21b-49f496e1371d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:10.563922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef067e-d863-4a62-b21b-49f496e1371d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:10.564203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef067e-d863-4a62-b21b-49f496e1371d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:10.565790 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 450/909] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:25:10.570789 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c1758-9bbc-495a-84ed-a9882b5698d3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:10.571090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c1758-9bbc-495a-84ed-a9882b5698d3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:10.571391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c1758-9bbc-495a-84ed-a9882b5698d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:10.579533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:10.591875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-967c1758-9bbc-495a-84ed-a9882b5698d3 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:10.593066 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-967c1758-9bbc-495a-84ed-a9882b5698d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:10.626117 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a40d2-09aa-4f65-b8b3-0de2523361af None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:10.671321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 451/910] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:10.672549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9436bf-f986-4e16-a0f9-35b75f87262d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:10.673398 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9436bf-f986-4e16-a0f9-35b75f87262d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:10.673902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9436bf-f986-4e16-a0f9-35b75f87262d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:10.674398 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9436bf-f986-4e16-a0f9-35b75f87262d None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:10.675294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9436bf-f986-4e16-a0f9-35b75f87262d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:10.676972 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9436bf-f986-4e16-a0f9-35b75f87262d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:10.684292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 460/911] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Aug 1 07:25:10 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:25:10.689438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c1758-9bbc-495a-84ed-a9882b5698d3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:10.801492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c1758-9bbc-495a-84ed-a9882b5698d3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:10.837921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c41e4-0ae8-42d3-b69d-84a2ab19a819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:10.838551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c41e4-0ae8-42d3-b69d-84a2ab19a819 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:10.838831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c41e4-0ae8-42d3-b69d-84a2ab19a819 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:10.839132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c41e4-0ae8-42d3-b69d-84a2ab19a819 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:10.839848 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c41e4-0ae8-42d3-b69d-84a2ab19a819 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:10.841221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c41e4-0ae8-42d3-b69d-84a2ab19a819 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:10.844845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:10.860681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 452/912] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:25:10 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:25:11.644475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a28a1-5e7d-48fc-808b-5b97d330e005 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:11.644995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a28a1-5e7d-48fc-808b-5b97d330e005 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:11.644995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a28a1-5e7d-48fc-808b-5b97d330e005 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:11.646666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 461/913] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:11.652149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a59648-b3c8-4d4e-904f-a09cf6027daf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:11.652976 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a59648-b3c8-4d4e-904f-a09cf6027daf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:11.653452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a59648-b3c8-4d4e-904f-a09cf6027daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:11.672369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-59a59648-b3c8-4d4e-904f-a09cf6027daf None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:11.673872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-59a59648-b3c8-4d4e-904f-a09cf6027daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:11.759367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 453/914] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:11.775830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a28a1-5e7d-48fc-808b-5b97d330e005 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:11.852747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:11.888237 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a28a1-5e7d-48fc-808b-5b97d330e005 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:11.925597 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527962c5-f43c-4066-b9e3-1dd41e85ca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:11.926250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527962c5-f43c-4066-b9e3-1dd41e85ca76 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:11.926535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527962c5-f43c-4066-b9e3-1dd41e85ca76 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:11.926815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527962c5-f43c-4066-b9e3-1dd41e85ca76 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:11.927475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527962c5-f43c-4066-b9e3-1dd41e85ca76 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:11.928904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527962c5-f43c-4066-b9e3-1dd41e85ca76 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-527962c5-f43c-4066-b9e3-1dd41e85ca76 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:11.938035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:11.939128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:11.940296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 462/915] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:25:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:11.953879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a59648-b3c8-4d4e-904f-a09cf6027daf None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:11.994218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aea30-fcd3-43a8-858d-aad6feacd181 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:11.994552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aea30-fcd3-43a8-858d-aad6feacd181 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:11.994789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aea30-fcd3-43a8-858d-aad6feacd181 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:11.996355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 463/916] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:25:12.001454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7f0b6-e6b6-4660-8e97-c2ed498fed5a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:12.001743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7f0b6-e6b6-4660-8e97-c2ed498fed5a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:12.002024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7f0b6-e6b6-4660-8e97-c2ed498fed5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:12.021919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-96f7f0b6-e6b6-4660-8e97-c2ed498fed5a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:12.023504 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-96f7f0b6-e6b6-4660-8e97-c2ed498fed5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:12.090406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a59648-b3c8-4d4e-904f-a09cf6027daf None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:12.103506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 464/917] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:12.117623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:12.119876 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7f0b6-e6b6-4660-8e97-c2ed498fed5a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:12.137597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71a49b-e9af-4a54-8a1e-47b2e62095f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:12.138576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71a49b-e9af-4a54-8a1e-47b2e62095f6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:12.139366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71a49b-e9af-4a54-8a1e-47b2e62095f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:12.139808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71a49b-e9af-4a54-8a1e-47b2e62095f6 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:12.140665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71a49b-e9af-4a54-8a1e-47b2e62095f6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:12.141142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71a49b-e9af-4a54-8a1e-47b2e62095f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:12.143161 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71a49b-e9af-4a54-8a1e-47b2e62095f6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:12.150775 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 454/918] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:25:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:12.159994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71a49b-e9af-4a54-8a1e-47b2e62095f6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:12.192428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:12.233730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7f0b6-e6b6-4660-8e97-c2ed498fed5a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:12.277102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1e3bf-aa40-4ba9-b293-b40878c1d519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:12.278411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1e3bf-aa40-4ba9-b293-b40878c1d519 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:12.278910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1e3bf-aa40-4ba9-b293-b40878c1d519 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:12.279652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1e3bf-aa40-4ba9-b293-b40878c1d519 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:12.289136 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71a49b-e9af-4a54-8a1e-47b2e62095f6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:12.294306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1e3bf-aa40-4ba9-b293-b40878c1d519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-93f1e3bf-aa40-4ba9-b293-b40878c1d519 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:12.300819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:12.301786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:12.302613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:12.302613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:12.302613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:12.302613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:12.302613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:12.302613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:12.302613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:12.302613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:12.302613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:12.302613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:12.302613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:25:12.302613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:12.309924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 465/919] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Aug 1 07:25:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:12.312486 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1e3bf-aa40-4ba9-b293-b40878c1d519 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:12.336091 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185f1eb-ae46-4d45-80aa-84604349b1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:12.337081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185f1eb-ae46-4d45-80aa-84604349b1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:12.337600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185f1eb-ae46-4d45-80aa-84604349b1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:12.338116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185f1eb-ae46-4d45-80aa-84604349b1c4 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:12.339104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185f1eb-ae46-4d45-80aa-84604349b1c4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:12.339634 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185f1eb-ae46-4d45-80aa-84604349b1c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:12.341295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185f1eb-ae46-4d45-80aa-84604349b1c4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:12.347306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 455/920] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Aug 1 07:25:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:12.354561 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185f1eb-ae46-4d45-80aa-84604349b1c4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:12.406929 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:12.431569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1e3bf-aa40-4ba9-b293-b40878c1d519 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:12.458059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:12.467003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185f1eb-ae46-4d45-80aa-84604349b1c4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:12.472040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56419e-700e-44a6-a46e-36d0ef30893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:12.472865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56419e-700e-44a6-a46e-36d0ef30893c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:12.473322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56419e-700e-44a6-a46e-36d0ef30893c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:12.473754 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56419e-700e-44a6-a46e-36d0ef30893c None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:12.474628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56419e-700e-44a6-a46e-36d0ef30893c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:12.475161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56419e-700e-44a6-a46e-36d0ef30893c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:12.477383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56419e-700e-44a6-a46e-36d0ef30893c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:12.487012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 466/921] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Aug 1 07:25:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:12.495313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56419e-700e-44a6-a46e-36d0ef30893c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:12.510449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e73c4-cb29-454c-9360-43562b54266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:12.511734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e73c4-cb29-454c-9360-43562b54266f None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:12.512193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e73c4-cb29-454c-9360-43562b54266f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:12.512957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e73c4-cb29-454c-9360-43562b54266f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:12.514130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e73c4-cb29-454c-9360-43562b54266f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:12.515880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e73c4-cb29-454c-9360-43562b54266f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:12.547182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 456/922] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:25:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:25:12.623241 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56419e-700e-44a6-a46e-36d0ef30893c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:12.669268 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bc738-d3b6-4ef3-a877-85acffb648fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:12.670622 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bc738-d3b6-4ef3-a877-85acffb648fb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:12.671107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bc738-d3b6-4ef3-a877-85acffb648fb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:12.671552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bc738-d3b6-4ef3-a877-85acffb648fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:12.684332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53bc738-d3b6-4ef3-a877-85acffb648fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:12.686129 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53bc738-d3b6-4ef3-a877-85acffb648fb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a53bc738-d3b6-4ef3-a877-85acffb648fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:12.703251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:12.704424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:12.705627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:12.705627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:12.705627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:12.705627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:12.707529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 467/923] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:12.713303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e73c4-cb29-454c-9360-43562b54266f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:12.822612 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e73c4-cb29-454c-9360-43562b54266f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:12.825398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:12.860991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48292cea-8162-4d5c-bc76-6beac9e61a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:12.861664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48292cea-8162-4d5c-bc76-6beac9e61a2c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:12.861955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48292cea-8162-4d5c-bc76-6beac9e61a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:12.862237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48292cea-8162-4d5c-bc76-6beac9e61a2c None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:12.862936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48292cea-8162-4d5c-bc76-6beac9e61a2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:12.863274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48292cea-8162-4d5c-bc76-6beac9e61a2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:12.864649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48292cea-8162-4d5c-bc76-6beac9e61a2c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:12.875630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f37a8-997a-4f63-a20f-a3f10182d013 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:12.875922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f37a8-997a-4f63-a20f-a3f10182d013 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:12.876199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f37a8-997a-4f63-a20f-a3f10182d013 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:12.877924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 468/924] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:12.880024 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 457/925] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:12.884933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfa7d3-d996-4052-849a-bf4cb82705cf None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:12.885290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfa7d3-d996-4052-849a-bf4cb82705cf None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:12.885662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfa7d3-d996-4052-849a-bf4cb82705cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:12.885814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48292cea-8162-4d5c-bc76-6beac9e61a2c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:12.905897 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-cccfa7d3-d996-4052-849a-bf4cb82705cf None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:12.907340 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cccfa7d3-d996-4052-849a-bf4cb82705cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:12.940814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:12.994311 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 469/926] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:13.001383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc4cd1-128e-4cb8-b2af-511b18d52f6e None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.002102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc4cd1-128e-4cb8-b2af-511b18d52f6e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.002381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48292cea-8162-4d5c-bc76-6beac9e61a2c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:13.003024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc4cd1-128e-4cb8-b2af-511b18d52f6e None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.013416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 470/927] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:12 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:13.014366 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f0026-a8ba-440d-8880-edb5ee22512b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.014651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f0026-a8ba-440d-8880-edb5ee22512b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.014925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f0026-a8ba-440d-8880-edb5ee22512b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.034954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2a3f0026-a8ba-440d-8880-edb5ee22512b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:13.036152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2a3f0026-a8ba-440d-8880-edb5ee22512b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:13.043868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e98aea-edc9-4f8d-ae0e-c8f2031dce6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:13.044511 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e98aea-edc9-4f8d-ae0e-c8f2031dce6f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.044792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e98aea-edc9-4f8d-ae0e-c8f2031dce6f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.045069 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e98aea-edc9-4f8d-ae0e-c8f2031dce6f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.052722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e98aea-edc9-4f8d-ae0e-c8f2031dce6f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:13.054512 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e98aea-edc9-4f8d-ae0e-c8f2031dce6f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-70e98aea-edc9-4f8d-ae0e-c8f2031dce6f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:25:13.066975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:13.068061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:13.069006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:13.069006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:13.069006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:25:13.069006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:25:13.069006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:25:13.069006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:25:13.069006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:25:13.069006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:25:13.069006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:25:13.069006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:13.069006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 458/928] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Aug 1 07:25:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:13.069006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e98aea-edc9-4f8d-ae0e-c8f2031dce6f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:13.097353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:13.123110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 471/929] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:13.123110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f0026-a8ba-440d-8880-edb5ee22512b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:13.187157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e98aea-edc9-4f8d-ae0e-c8f2031dce6f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:13.226397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c000ccf-9601-4eef-8f0a-fc7671a7921d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:13.227248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c000ccf-9601-4eef-8f0a-fc7671a7921d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.227534 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c000ccf-9601-4eef-8f0a-fc7671a7921d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.227809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c000ccf-9601-4eef-8f0a-fc7671a7921d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.228282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:13.228568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c000ccf-9601-4eef-8f0a-fc7671a7921d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:13.230752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c000ccf-9601-4eef-8f0a-fc7671a7921d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:13.234849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f0026-a8ba-440d-8880-edb5ee22512b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7c000ccf-9601-4eef-8f0a-fc7671a7921d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:13.241514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:13.242468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:13.248652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 459/930] 10.222.0.40 () {56 vars in 1132 bytes} [Mon Aug 1 07:25:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:13.249537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c000ccf-9601-4eef-8f0a-fc7671a7921d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:13.270555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb964d-b50f-4d70-a91e-767a897d45f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:13.271219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb964d-b50f-4d70-a91e-767a897d45f4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.271504 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb964d-b50f-4d70-a91e-767a897d45f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.271784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb964d-b50f-4d70-a91e-767a897d45f4 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.273137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb964d-b50f-4d70-a91e-767a897d45f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:13.273670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb964d-b50f-4d70-a91e-767a897d45f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:13.275307 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb964d-b50f-4d70-a91e-767a897d45f4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:13.298837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 472/931] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Aug 1 07:25:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:13.298837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb964d-b50f-4d70-a91e-767a897d45f4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:13.351457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:13.364998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c000ccf-9601-4eef-8f0a-fc7671a7921d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:13.401077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe419c-a243-4c2d-82bb-f46f82dc7565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:13.401717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe419c-a243-4c2d-82bb-f46f82dc7565 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.402002 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe419c-a243-4c2d-82bb-f46f82dc7565 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.402286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe419c-a243-4c2d-82bb-f46f82dc7565 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.402914 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe419c-a243-4c2d-82bb-f46f82dc7565 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:13.404308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe419c-a243-4c2d-82bb-f46f82dc7565 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:13.407170 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb964d-b50f-4d70-a91e-767a897d45f4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e2fe419c-a243-4c2d-82bb-f46f82dc7565 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:13.413527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:13.414543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:13.415851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 460/932] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Aug 1 07:25:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:13.416834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe419c-a243-4c2d-82bb-f46f82dc7565 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:13.449928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e2bab-1f57-4ef2-af60-4c9a93a4395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:13.450917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e2bab-1f57-4ef2-af60-4c9a93a4395b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.451413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e2bab-1f57-4ef2-af60-4c9a93a4395b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.451687 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e2bab-1f57-4ef2-af60-4c9a93a4395b None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.452359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e2bab-1f57-4ef2-af60-4c9a93a4395b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:13.452668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e2bab-1f57-4ef2-af60-4c9a93a4395b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:13.454021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e2bab-1f57-4ef2-af60-4c9a93a4395b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:13.463330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 473/933] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Aug 1 07:25:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:13.463330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e2bab-1f57-4ef2-af60-4c9a93a4395b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:13.482806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:13.529089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe419c-a243-4c2d-82bb-f46f82dc7565 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:13.562212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf5d88-68e0-4413-9390-7818c97dfe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:13.562844 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf5d88-68e0-4413-9390-7818c97dfe14 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.563174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf5d88-68e0-4413-9390-7818c97dfe14 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.563453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf5d88-68e0-4413-9390-7818c97dfe14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.570214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e2bab-1f57-4ef2-af60-4c9a93a4395b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:13.574241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf5d88-68e0-4413-9390-7818c97dfe14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:13.575710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf5d88-68e0-4413-9390-7818c97dfe14 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7dbf5d88-68e0-4413-9390-7818c97dfe14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:13.590552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:13.591666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:13.592746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:13.592746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:13.592746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:13.592746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:13.592746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 461/934] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:13.592746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf5d88-68e0-4413-9390-7818c97dfe14 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:13.597680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:13.615507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6d746-fc6a-4326-b977-0402efe51b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:13.616752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6d746-fc6a-4326-b977-0402efe51b23 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.616947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6d746-fc6a-4326-b977-0402efe51b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.617327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6d746-fc6a-4326-b977-0402efe51b23 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.617996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6d746-fc6a-4326-b977-0402efe51b23 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:13.618296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6d746-fc6a-4326-b977-0402efe51b23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:13.619701 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6d746-fc6a-4326-b977-0402efe51b23 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:13.629332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 474/935] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Aug 1 07:25:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:13.629332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6d746-fc6a-4326-b977-0402efe51b23 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:13.707570 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf5d88-68e0-4413-9390-7818c97dfe14 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:13.715960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:13.738792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6d746-fc6a-4326-b977-0402efe51b23 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:13.745427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24f77b-1509-42dc-9dae-e1c4694a6588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:13.746157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24f77b-1509-42dc-9dae-e1c4694a6588 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.746353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24f77b-1509-42dc-9dae-e1c4694a6588 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.746629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24f77b-1509-42dc-9dae-e1c4694a6588 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.747360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24f77b-1509-42dc-9dae-e1c4694a6588 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:13.748283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24f77b-1509-42dc-9dae-e1c4694a6588 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:13.750101 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24f77b-1509-42dc-9dae-e1c4694a6588 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:13.756031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 462/936] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Aug 1 07:25:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:13.761952 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24f77b-1509-42dc-9dae-e1c4694a6588 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:13.777343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e717f9e-793a-411d-8749-d565af83d409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:13.777970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e717f9e-793a-411d-8749-d565af83d409 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.778255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e717f9e-793a-411d-8749-d565af83d409 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.778540 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e717f9e-793a-411d-8749-d565af83d409 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.779265 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e717f9e-793a-411d-8749-d565af83d409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:13.779679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e717f9e-793a-411d-8749-d565af83d409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:13.780944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e717f9e-793a-411d-8749-d565af83d409 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:13.795425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 475/937] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:13.796226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e717f9e-793a-411d-8749-d565af83d409 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:13.840125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:13.875870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24f77b-1509-42dc-9dae-e1c4694a6588 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:13.905354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e717f9e-793a-411d-8749-d565af83d409 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:13.911397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4a840-e93a-4d2d-a216-8972dd066a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:13.912016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4a840-e93a-4d2d-a216-8972dd066a93 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.912303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4a840-e93a-4d2d-a216-8972dd066a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.912585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4a840-e93a-4d2d-a216-8972dd066a93 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.914065 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4a840-e93a-4d2d-a216-8972dd066a93 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:13.914386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4a840-e93a-4d2d-a216-8972dd066a93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:13.915946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4a840-e93a-4d2d-a216-8972dd066a93 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:13.921133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 463/938] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Aug 1 07:25:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:13.921719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4a840-e93a-4d2d-a216-8972dd066a93 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:13.950198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263f4fe-7d16-44d2-86dc-a3da7a06a99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:13.950831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263f4fe-7d16-44d2-86dc-a3da7a06a99e None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:13.951551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263f4fe-7d16-44d2-86dc-a3da7a06a99e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:13.951841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263f4fe-7d16-44d2-86dc-a3da7a06a99e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:13.952800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263f4fe-7d16-44d2-86dc-a3da7a06a99e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:13.954208 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263f4fe-7d16-44d2-86dc-a3da7a06a99e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:13.967241 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:13.989321 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 476/939] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:25:13 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:25:13.990329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263f4fe-7d16-44d2-86dc-a3da7a06a99e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:14.056230 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4a840-e93a-4d2d-a216-8972dd066a93 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:14.098755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:14.114592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ca583-95ab-4dea-af92-88291de58523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:14.116144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ca583-95ab-4dea-af92-88291de58523 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:14.116639 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ca583-95ab-4dea-af92-88291de58523 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:14.117153 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ca583-95ab-4dea-af92-88291de58523 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:14.126673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ca583-95ab-4dea-af92-88291de58523 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:14.128190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ca583-95ab-4dea-af92-88291de58523 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:14.137406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263f4fe-7d16-44d2-86dc-a3da7a06a99e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-232ca583-95ab-4dea-af92-88291de58523 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:25:14.142100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:14.143019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:14.143964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:25:14.143964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:25:14.143964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:25:14.143964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:25:14.143964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:25:14.143964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:25:14.143964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:14.143964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:14.147147 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 464/940] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:14.149468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ca583-95ab-4dea-af92-88291de58523 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:14.176988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ddf1b-3019-470a-af70-e2b213312304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:14.177620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ddf1b-3019-470a-af70-e2b213312304 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:14.177897 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ddf1b-3019-470a-af70-e2b213312304 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:14.178172 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ddf1b-3019-470a-af70-e2b213312304 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:14.179504 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ddf1b-3019-470a-af70-e2b213312304 None admin] RBAC: Adding query filter params (interface=public, service_id=a67d65cbf1794454857e439406d459fd, region_id=RegionOne) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:14.179819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ddf1b-3019-470a-af70-e2b213312304 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:14.181215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ddf1b-3019-470a-af70-e2b213312304 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:14.189801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 477/941] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Aug 1 07:25:13 2022] GET /identity/v3/endpoints?service_id=a67d65cbf1794454857e439406d459fd&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:14.196041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ddf1b-3019-470a-af70-e2b213312304 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:14.271796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:14.278034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ca583-95ab-4dea-af92-88291de58523 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:14.319865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ddf1b-3019-470a-af70-e2b213312304 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:14.326739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae4582-c65d-413a-ae23-c5210c3e3a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:14.328192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae4582-c65d-413a-ae23-c5210c3e3a21 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:14.328704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae4582-c65d-413a-ae23-c5210c3e3a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:14.329635 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae4582-c65d-413a-ae23-c5210c3e3a21 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:14.330639 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae4582-c65d-413a-ae23-c5210c3e3a21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:14.331615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae4582-c65d-413a-ae23-c5210c3e3a21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:14.333768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae4582-c65d-413a-ae23-c5210c3e3a21 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:14.358019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 465/942] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:25:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:14.359828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae4582-c65d-413a-ae23-c5210c3e3a21 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:14.361630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1344e-8b8d-4a19-b15c-d004a7871098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:14.362255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1344e-8b8d-4a19-b15c-d004a7871098 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:14.362536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1344e-8b8d-4a19-b15c-d004a7871098 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:14.362807 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1344e-8b8d-4a19-b15c-d004a7871098 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:14.363514 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1344e-8b8d-4a19-b15c-d004a7871098 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:14.364881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1344e-8b8d-4a19-b15c-d004a7871098 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:14.372098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 478/943] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Aug 1 07:25:14 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:25:14.420325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:14.480688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae4582-c65d-413a-ae23-c5210c3e3a21 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:14.521042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bb5e9-687e-4e77-9e81-349366a6d9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:14.522192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bb5e9-687e-4e77-9e81-349366a6d9e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:14.523256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bb5e9-687e-4e77-9e81-349366a6d9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:14.523752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bb5e9-687e-4e77-9e81-349366a6d9e2 None admin] PATH_INFO: `/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/8340fafe48af45c8b87a97aaac637bd5/roles/d1e0bfc9cbb14751b3daabc5e2de9813` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:14.527240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bb5e9-687e-4e77-9e81-349366a6d9e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ddf7d6f2424a85a741d017497b49c9, user_id=8340fafe48af45c8b87a97aaac637bd5, role_id=d1e0bfc9cbb14751b3daabc5e2de9813)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:14.528913 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bb5e9-687e-4e77-9e81-349366a6d9e2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:14.551164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 466/944] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:25:14 2022] PUT /identity/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/8340fafe48af45c8b87a97aaac637bd5/roles/d1e0bfc9cbb14751b3daabc5e2de9813 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:25:14.667866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed7ab8-0fd7-4779-b2f0-db20a1f2dbb0 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:14.668267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed7ab8-0fd7-4779-b2f0-db20a1f2dbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:14.668438 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed7ab8-0fd7-4779-b2f0-db20a1f2dbb0 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:14.670110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 479/945] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:14.675904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0bd40-19c7-4fdb-a9da-9cbb64752f38 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:14.676181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0bd40-19c7-4fdb-a9da-9cbb64752f38 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:14.676462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0bd40-19c7-4fdb-a9da-9cbb64752f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:14.699860 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-57d0bd40-19c7-4fdb-a9da-9cbb64752f38 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:14.700515 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-57d0bd40-19c7-4fdb-a9da-9cbb64752f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:14.775653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 467/946] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:14.799089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed7ab8-0fd7-4779-b2f0-db20a1f2dbb0 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:14.910797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed7ab8-0fd7-4779-b2f0-db20a1f2dbb0 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:14.936168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:14.948341 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73681053-d819-4215-81d8-d67f98b8dba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:14.948966 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73681053-d819-4215-81d8-d67f98b8dba8 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:14.949249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73681053-d819-4215-81d8-d67f98b8dba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:14.949525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73681053-d819-4215-81d8-d67f98b8dba8 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:14.950229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73681053-d819-4215-81d8-d67f98b8dba8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:14.952804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73681053-d819-4215-81d8-d67f98b8dba8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:14.955545 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: WARNING keystone.server.flask.common [None req-73681053-d819-4215-81d8-d67f98b8dba8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 07:25:14.982697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.sql.core [None req-73681053-d819-4215-81d8-d67f98b8dba8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 07:25:14.982697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 07:25:14.982697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [parameters: {'id': '9b429564f89d405c8524f15b281ebfad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 07:25:14.982697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80159) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-73681053-d819-4215-81d8-d67f98b8dba8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9b429564f89d405c8524f15b281ebfad.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9b429564f89d405c8524f15b281ebfad. Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application conn.query(q) Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 07:25:15.001364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application result.read() Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 07:25:15.002222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application next(self.gen) Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application yield resource Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application next(self.gen) Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise self.value Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application session.commit() Aug 01 07:25:15.003219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self.session.flush() Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application compat.raise_( Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 07:25:15.004104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 07:25:15.004984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application conn.query(q) Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application result.read() Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 07:25:15.005758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [parameters: {'id': '9b429564f89d405c8524f15b281ebfad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:15.006603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:15.009551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:15.011613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:15.011613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 07:25:15.011613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 07:25:15.011613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:15.011613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:15.011613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 07:25:15.011613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 07:25:15.011613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9b429564f89d405c8524f15b281ebfad. Aug 01 07:25:15.011613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:15.011613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 480/947] 10.222.0.40 () {60 vars in 1140 bytes} [Mon Aug 1 07:25:14 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:15.012852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0bd40-19c7-4fdb-a9da-9cbb64752f38 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:15.061896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:15.126446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0bd40-19c7-4fdb-a9da-9cbb64752f38 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:15.163800 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31176ad7-18ae-4e44-8bb2-7d3dde40b206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:15.164454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31176ad7-18ae-4e44-8bb2-7d3dde40b206 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:15.164741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31176ad7-18ae-4e44-8bb2-7d3dde40b206 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:15.165022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31176ad7-18ae-4e44-8bb2-7d3dde40b206 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:15.172184 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31176ad7-18ae-4e44-8bb2-7d3dde40b206 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:15.173679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31176ad7-18ae-4e44-8bb2-7d3dde40b206 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-31176ad7-18ae-4e44-8bb2-7d3dde40b206 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:25:15.187674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:15.189743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:15.191823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:25:15.191823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:25:15.191823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:25:15.191823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:25:15.191823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:25:15.191823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:25:15.191823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:15.191823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:15.191823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 468/948] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:15.195473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73681053-d819-4215-81d8-d67f98b8dba8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:15.307095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73681053-d819-4215-81d8-d67f98b8dba8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:15.343409 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f24314-b10e-4f0d-a74d-24f3edf9f2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:15.343965 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f24314-b10e-4f0d-a74d-24f3edf9f2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:15.344249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f24314-b10e-4f0d-a74d-24f3edf9f2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:15.344583 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f24314-b10e-4f0d-a74d-24f3edf9f2f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:15.345356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f24314-b10e-4f0d-a74d-24f3edf9f2f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:15.345666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f24314-b10e-4f0d-a74d-24f3edf9f2f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:15.347330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f24314-b10e-4f0d-a74d-24f3edf9f2f5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:15.368901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 481/949] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:25:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:16.124797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945d658-a0a4-4f86-ab4a-5888644d66c3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:16.125339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945d658-a0a4-4f86-ab4a-5888644d66c3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:16.125339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945d658-a0a4-4f86-ab4a-5888644d66c3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:16.126956 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 469/950] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:16.133354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b011c-479d-4db5-9e69-8c0da3b418b9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:16.133775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b011c-479d-4db5-9e69-8c0da3b418b9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:16.134175 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b011c-479d-4db5-9e69-8c0da3b418b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:16.153414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-363b011c-479d-4db5-9e69-8c0da3b418b9 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:16.154950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-363b011c-479d-4db5-9e69-8c0da3b418b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:16.192921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:16.229579 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 482/951] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:16.251841 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945d658-a0a4-4f86-ab4a-5888644d66c3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:16.355441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:16.360336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945d658-a0a4-4f86-ab4a-5888644d66c3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:16.396689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736c2c9-fa59-479e-b03d-18054f10938d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:16.397383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736c2c9-fa59-479e-b03d-18054f10938d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:16.397672 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736c2c9-fa59-479e-b03d-18054f10938d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:16.398138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736c2c9-fa59-479e-b03d-18054f10938d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:16.398771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736c2c9-fa59-479e-b03d-18054f10938d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:16.400185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736c2c9-fa59-479e-b03d-18054f10938d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a736c2c9-fa59-479e-b03d-18054f10938d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:16.409572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:16.410599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:16.411905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 470/952] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:16.418317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b011c-479d-4db5-9e69-8c0da3b418b9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:16.518732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f182f7-2fe6-42bd-b4e5-a9db45346fd6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:16.518935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f182f7-2fe6-42bd-b4e5-a9db45346fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:16.519256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f182f7-2fe6-42bd-b4e5-a9db45346fd6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:16.523124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 471/953] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:25:16.526563 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff8bf7-46fe-46fb-8923-b679f3c30672 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:16.526966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff8bf7-46fe-46fb-8923-b679f3c30672 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:16.527370 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff8bf7-46fe-46fb-8923-b679f3c30672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:16.528081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b011c-479d-4db5-9e69-8c0da3b418b9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:16.546509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-10ff8bf7-46fe-46fb-8923-b679f3c30672 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:16.547995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-10ff8bf7-46fe-46fb-8923-b679f3c30672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:16.569414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974017e-9c9c-4f6c-a273-da820e82bcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:16.570284 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974017e-9c9c-4f6c-a273-da820e82bcc2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:16.570722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974017e-9c9c-4f6c-a273-da820e82bcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:16.571209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974017e-9c9c-4f6c-a273-da820e82bcc2 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:16.572061 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974017e-9c9c-4f6c-a273-da820e82bcc2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:16.572521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974017e-9c9c-4f6c-a273-da820e82bcc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:16.574045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974017e-9c9c-4f6c-a273-da820e82bcc2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:16.579637 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 483/954] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:25:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:16.586243 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974017e-9c9c-4f6c-a273-da820e82bcc2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:16.625759 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:16.639398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 472/955] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:16.657607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff8bf7-46fe-46fb-8923-b679f3c30672 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:16.702316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974017e-9c9c-4f6c-a273-da820e82bcc2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:16.739044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16220b80-f0bd-487f-a934-de45f0ee2b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:16.739996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16220b80-f0bd-487f-a934-de45f0ee2b66 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:16.740466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16220b80-f0bd-487f-a934-de45f0ee2b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:16.741510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16220b80-f0bd-487f-a934-de45f0ee2b66 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:16.742204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16220b80-f0bd-487f-a934-de45f0ee2b66 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:16.742510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16220b80-f0bd-487f-a934-de45f0ee2b66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:16.743908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16220b80-f0bd-487f-a934-de45f0ee2b66 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:16.748627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 484/956] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:25:16 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:16.749770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864e3f2-43db-4c6c-aad7-b92cc42e62fc None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:16.750062 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864e3f2-43db-4c6c-aad7-b92cc42e62fc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:16.750344 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864e3f2-43db-4c6c-aad7-b92cc42e62fc None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:16.755811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 485/957] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:16.757305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617e7c7-1704-4eac-996f-0035cf3d443a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:16.757596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617e7c7-1704-4eac-996f-0035cf3d443a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:16.757873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617e7c7-1704-4eac-996f-0035cf3d443a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:16.768964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:16.780788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6617e7c7-1704-4eac-996f-0035cf3d443a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:16.782030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6617e7c7-1704-4eac-996f-0035cf3d443a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:16.782582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff8bf7-46fe-46fb-8923-b679f3c30672 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:16.835505 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9466fa6-3ed6-4e58-945e-9511fe39d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:16.838317 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9466fa6-3ed6-4e58-945e-9511fe39d8fa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:16.838607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9466fa6-3ed6-4e58-945e-9511fe39d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:16.838934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9466fa6-3ed6-4e58-945e-9511fe39d8fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:16.840050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9466fa6-3ed6-4e58-945e-9511fe39d8fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:16.841475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9466fa6-3ed6-4e58-945e-9511fe39d8fa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f9466fa6-3ed6-4e58-945e-9511fe39d8fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:16.852174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:16.853377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:16.854283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:16.858918 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 473/958] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Aug 1 07:25:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:16.859978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9466fa6-3ed6-4e58-945e-9511fe39d8fa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:16.883296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 486/959] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:16.905545 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617e7c7-1704-4eac-996f-0035cf3d443a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:16.957309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:16.990175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9466fa6-3ed6-4e58-945e-9511fe39d8fa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:17.029519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa5421-5a7b-4706-bfae-fe21f4ed77aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:17.030221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa5421-5a7b-4706-bfae-fe21f4ed77aa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:17.030602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa5421-5a7b-4706-bfae-fe21f4ed77aa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:17.030981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa5421-5a7b-4706-bfae-fe21f4ed77aa None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:17.031846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa5421-5a7b-4706-bfae-fe21f4ed77aa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:17.032254 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa5421-5a7b-4706-bfae-fe21f4ed77aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:17.033721 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa5421-5a7b-4706-bfae-fe21f4ed77aa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:17.039574 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 474/960] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Aug 1 07:25:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:17.044791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa5421-5a7b-4706-bfae-fe21f4ed77aa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:17.062829 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617e7c7-1704-4eac-996f-0035cf3d443a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:17.071523 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:17.099959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd7ab2-4ba3-4bb4-9d39-fbbe5431c4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:17.100820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd7ab2-4ba3-4bb4-9d39-fbbe5431c4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:17.101200 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd7ab2-4ba3-4bb4-9d39-fbbe5431c4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:17.101674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd7ab2-4ba3-4bb4-9d39-fbbe5431c4f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:17.112781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd7ab2-4ba3-4bb4-9d39-fbbe5431c4f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-e6cd7ab2-4ba3-4bb4-9d39-fbbe5431c4f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:17.118299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:17.119456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:17.120499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:17.120499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:17.120499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:17.120499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:17.120499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:17.120499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:17.120499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:17.120499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:17.120499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:17.120499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:17.120499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:25:17.120499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:17.122491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 487/961] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Aug 1 07:25:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:17.128548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd7ab2-4ba3-4bb4-9d39-fbbe5431c4f3 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:17.167252 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa5421-5a7b-4706-bfae-fe21f4ed77aa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:17.211967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca059fd-8083-4d19-a9c2-cd06c49fb51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:17.213030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca059fd-8083-4d19-a9c2-cd06c49fb51c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:17.213417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca059fd-8083-4d19-a9c2-cd06c49fb51c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:17.213796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca059fd-8083-4d19-a9c2-cd06c49fb51c None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:17.214592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca059fd-8083-4d19-a9c2-cd06c49fb51c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:17.215179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca059fd-8083-4d19-a9c2-cd06c49fb51c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:17.217489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca059fd-8083-4d19-a9c2-cd06c49fb51c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:17.223600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 475/962] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Aug 1 07:25:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:17.234985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca059fd-8083-4d19-a9c2-cd06c49fb51c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:17.247814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:17.266296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd7ab2-4ba3-4bb4-9d39-fbbe5431c4f3 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:17.316100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc20ff-f5cf-46de-9b04-4d2215c7c7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:17.316978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc20ff-f5cf-46de-9b04-4d2215c7c7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:17.317560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc20ff-f5cf-46de-9b04-4d2215c7c7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:17.317993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc20ff-f5cf-46de-9b04-4d2215c7c7c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:17.319819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc20ff-f5cf-46de-9b04-4d2215c7c7c7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:17.320300 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc20ff-f5cf-46de-9b04-4d2215c7c7c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:17.321823 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc20ff-f5cf-46de-9b04-4d2215c7c7c7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:17.332972 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 488/963] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Aug 1 07:25:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:17.335773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:17.339138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc20ff-f5cf-46de-9b04-4d2215c7c7c7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:17.356059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca059fd-8083-4d19-a9c2-cd06c49fb51c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:17.392051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ef409-4d70-48c1-af2a-c19590e7ca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:17.392815 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ef409-4d70-48c1-af2a-c19590e7ca9a None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:17.393214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ef409-4d70-48c1-af2a-c19590e7ca9a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:17.393589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ef409-4d70-48c1-af2a-c19590e7ca9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:17.394393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ef409-4d70-48c1-af2a-c19590e7ca9a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:17.396233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ef409-4d70-48c1-af2a-c19590e7ca9a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:17.427094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 476/964] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:25:17 2022] POST /identity/v3/endpoints => generated 349 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:25:17.484780 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc20ff-f5cf-46de-9b04-4d2215c7c7c7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:17.534435 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340146a-f103-45f8-b0a4-2fe9217a7420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:17.534888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6c329-3cfe-4f24-9fe3-9a63029e6e93 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:17.535131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340146a-f103-45f8-b0a4-2fe9217a7420 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:17.535234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6c329-3cfe-4f24-9fe3-9a63029e6e93 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:17.535420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340146a-f103-45f8-b0a4-2fe9217a7420 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:17.535515 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6c329-3cfe-4f24-9fe3-9a63029e6e93 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:17.535658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340146a-f103-45f8-b0a4-2fe9217a7420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:17.537051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 477/965] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:25:17.542830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab8c3a-fa5c-4610-acfa-ed2c7c5738c3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:17.543167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab8c3a-fa5c-4610-acfa-ed2c7c5738c3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:17.543452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab8c3a-fa5c-4610-acfa-ed2c7c5738c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:17.549832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340146a-f103-45f8-b0a4-2fe9217a7420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:17.551412 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340146a-f103-45f8-b0a4-2fe9217a7420 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:17.558923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:17.567078 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-67ab8c3a-fa5c-4610-acfa-ed2c7c5738c3 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9340146a-f103-45f8-b0a4-2fe9217a7420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:17.568506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:17.570197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:17.571096 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:17.571096 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:17.571096 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:17.571096 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:17.571341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-67ab8c3a-fa5c-4610-acfa-ed2c7c5738c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:17.571999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 489/966] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:17.576152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340146a-f103-45f8-b0a4-2fe9217a7420 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:17.656034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 478/967] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:17.681898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab8c3a-fa5c-4610-acfa-ed2c7c5738c3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:17.693080 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340146a-f103-45f8-b0a4-2fe9217a7420 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:17.727631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b005b-25f0-4ab8-995e-f1f8ab818e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:17.728701 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b005b-25f0-4ab8-995e-f1f8ab818e44 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:17.729202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b005b-25f0-4ab8-995e-f1f8ab818e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:17.729707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b005b-25f0-4ab8-995e-f1f8ab818e44 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:17.730638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b005b-25f0-4ab8-995e-f1f8ab818e44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:17.731182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b005b-25f0-4ab8-995e-f1f8ab818e44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:17.732775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b005b-25f0-4ab8-995e-f1f8ab818e44 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:17.746799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 490/968] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:17.751891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b005b-25f0-4ab8-995e-f1f8ab818e44 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:17.767210 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:17.805819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab8c3a-fa5c-4610-acfa-ed2c7c5738c3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:17.830333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:17.844492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838484b-366e-4881-a346-b30454249ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:17.845487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838484b-366e-4881-a346-b30454249ac6 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:17.846009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838484b-366e-4881-a346-b30454249ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:17.846489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838484b-366e-4881-a346-b30454249ac6 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:17.847436 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838484b-366e-4881-a346-b30454249ac6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:17.849096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838484b-366e-4881-a346-b30454249ac6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:17.852525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: WARNING keystone.server.flask.common [None req-9838484b-366e-4881-a346-b30454249ac6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 07:25:17.867117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.sql.core [None req-9838484b-366e-4881-a346-b30454249ac6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 07:25:17.867117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 07:25:17.867117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [parameters: {'user_id': 'c64dff51940245e6abd2e4c0aa854cd9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 07:25:17.867117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80160) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9838484b-366e-4881-a346-b30454249ac6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Aug 01 07:25:17.875357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 07:25:17.876239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application yield resource Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise self.value Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application session.commit() Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 07:25:17.877214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.flush() Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application compat.raise_( Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 07:25:17.878110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 07:25:17.879829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 07:25:17.881846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c64dff51940245e6abd2e4c0aa854cd9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:17.884242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 07:25:17.886093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 07:25:17.888356 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 07:25:17.888356 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 07:25:17.888356 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 07:25:17.888356 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 07:25:17.888356 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 07:25:17.888356 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:17.889812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b005b-25f0-4ab8-995e-f1f8ab818e44 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:17.891566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 479/969] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Aug 1 07:25:17 2022] POST /identity/v3/users => generated 160 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:17.897350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838484b-366e-4881-a346-b30454249ac6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:17.931368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde21d61-18af-48fd-b703-7bdd95b79d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:17.932930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde21d61-18af-48fd-b703-7bdd95b79d3b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:17.933716 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde21d61-18af-48fd-b703-7bdd95b79d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:17.934195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde21d61-18af-48fd-b703-7bdd95b79d3b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:17.945577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde21d61-18af-48fd-b703-7bdd95b79d3b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:17.947394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde21d61-18af-48fd-b703-7bdd95b79d3b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-bde21d61-18af-48fd-b703-7bdd95b79d3b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:25:17.968316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:17.969315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:17.970280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:17.970280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:17.970280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:25:17.970280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:25:17.970280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:25:17.970280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:25:17.970280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:25:17.970280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:25:17.970280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 07:25:17.970280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:17.974908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 491/970] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Aug 1 07:25:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:17.976310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde21d61-18af-48fd-b703-7bdd95b79d3b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:18.042109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838484b-366e-4881-a346-b30454249ac6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:18.076865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:18.084436 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ed874-3bdd-4d09-92ee-108ffca507e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:18.085806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ed874-3bdd-4d09-92ee-108ffca507e7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:18.086398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ed874-3bdd-4d09-92ee-108ffca507e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:18.086748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ed874-3bdd-4d09-92ee-108ffca507e7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:18.091105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde21d61-18af-48fd-b703-7bdd95b79d3b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:18.095031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ed874-3bdd-4d09-92ee-108ffca507e7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:18.096769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ed874-3bdd-4d09-92ee-108ffca507e7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:18.110070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-264ed874-3bdd-4d09-92ee-108ffca507e7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:25:18.112389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:18.113363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:18.115308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:18.115308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:18.115308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:25:18.115308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:25:18.115308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:25:18.115308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:25:18.115308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:25:18.115308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:25:18.115308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 07:25:18.115308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:18.115308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 480/971] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:25:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:18.120536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ed874-3bdd-4d09-92ee-108ffca507e7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:18.132982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a32d2-eaa4-4945-b8c0-d6c8b6dacb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:18.134128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a32d2-eaa4-4945-b8c0-d6c8b6dacb21 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:18.134600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a32d2-eaa4-4945-b8c0-d6c8b6dacb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:18.135052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a32d2-eaa4-4945-b8c0-d6c8b6dacb21 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:18.136051 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a32d2-eaa4-4945-b8c0-d6c8b6dacb21 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:18.136593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a32d2-eaa4-4945-b8c0-d6c8b6dacb21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:18.138209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a32d2-eaa4-4945-b8c0-d6c8b6dacb21 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:18.157238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 492/972] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Aug 1 07:25:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:18.162969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a32d2-eaa4-4945-b8c0-d6c8b6dacb21 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:18.254280 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ed874-3bdd-4d09-92ee-108ffca507e7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:18.293560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a32d2-eaa4-4945-b8c0-d6c8b6dacb21 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:18.299867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ab538-aa6a-439b-ae34-667548135eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:18.300488 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ab538-aa6a-439b-ae34-667548135eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:18.300767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ab538-aa6a-439b-ae34-667548135eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:18.301042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ab538-aa6a-439b-ae34-667548135eb8 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:18.301813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ab538-aa6a-439b-ae34-667548135eb8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:18.302119 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ab538-aa6a-439b-ae34-667548135eb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:18.304785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ab538-aa6a-439b-ae34-667548135eb8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:18.334432 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9b90d-5863-4c8a-b0c9-8543845455ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:18.335759 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9b90d-5863-4c8a-b0c9-8543845455ce None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:18.336251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9b90d-5863-4c8a-b0c9-8543845455ce None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:18.336749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9b90d-5863-4c8a-b0c9-8543845455ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:18.357193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 481/973] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Aug 1 07:25:18 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:18.395658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:18.424252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9b90d-5863-4c8a-b0c9-8543845455ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:18.426112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9b90d-5863-4c8a-b0c9-8543845455ce None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:18.539168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 493/974] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:18 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:25:18.546269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ab538-aa6a-439b-ae34-667548135eb8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:18.602951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:18.658786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ab538-aa6a-439b-ae34-667548135eb8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:18.694963 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556e454-0114-4476-9923-37ad2594d882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:18.695623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556e454-0114-4476-9923-37ad2594d882 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:18.695906 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556e454-0114-4476-9923-37ad2594d882 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:18.696189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556e454-0114-4476-9923-37ad2594d882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:18.706499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556e454-0114-4476-9923-37ad2594d882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:18.707953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556e454-0114-4476-9923-37ad2594d882 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4556e454-0114-4476-9923-37ad2594d882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:18.723102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:18.724067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:18.730152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:18.730152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:18.730152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:18.730152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:18.730152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 482/975] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:18.731796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9b90d-5863-4c8a-b0c9-8543845455ce None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:18.810756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:18.835660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9b90d-5863-4c8a-b0c9-8543845455ce None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:18.872273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374da6b-f641-44ab-bd18-b10e0f42c561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:18.872480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bec58-b53b-4b08-9dbc-36bfdedecd1a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:18.872480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bec58-b53b-4b08-9dbc-36bfdedecd1a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:18.872797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bec58-b53b-4b08-9dbc-36bfdedecd1a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:18.872909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374da6b-f641-44ab-bd18-b10e0f42c561 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:18.873094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374da6b-f641-44ab-bd18-b10e0f42c561 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:18.873375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374da6b-f641-44ab-bd18-b10e0f42c561 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:18.874250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374da6b-f641-44ab-bd18-b10e0f42c561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:18.874359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 483/976] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:25:18.874434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374da6b-f641-44ab-bd18-b10e0f42c561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:18.875814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374da6b-f641-44ab-bd18-b10e0f42c561 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:18.878687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5b4a5-1a5a-4f1e-ba3a-7ad614555c8b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:18.878970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5b4a5-1a5a-4f1e-ba3a-7ad614555c8b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:18.879273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5b4a5-1a5a-4f1e-ba3a-7ad614555c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:18.889711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 494/977] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:18.897291 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374da6b-f641-44ab-bd18-b10e0f42c561 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:18.898790 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9de5b4a5-1a5a-4f1e-ba3a-7ad614555c8b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:18.900335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9de5b4a5-1a5a-4f1e-ba3a-7ad614555c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:18.983347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 484/978] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:18.995933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5b4a5-1a5a-4f1e-ba3a-7ad614555c8b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:19.026925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374da6b-f641-44ab-bd18-b10e0f42c561 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:19.027141 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:19.064486 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc24ed6-dfca-4d5b-ac76-1e28b97ef71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:19.065128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc24ed6-dfca-4d5b-ac76-1e28b97ef71d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:19.065414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc24ed6-dfca-4d5b-ac76-1e28b97ef71d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:19.065696 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc24ed6-dfca-4d5b-ac76-1e28b97ef71d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:19.068774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:19.075411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc24ed6-dfca-4d5b-ac76-1e28b97ef71d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:19.076878 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc24ed6-dfca-4d5b-ac76-1e28b97ef71d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-afc24ed6-dfca-4d5b-ac76-1e28b97ef71d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:25:19.089449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:19.090349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:19.091272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:25:19.091272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:25:19.091272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:25:19.091272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:25:19.091272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:25:19.091272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:25:19.091272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:19.091272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:19.095433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 495/979] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:19.096261 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc24ed6-dfca-4d5b-ac76-1e28b97ef71d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:19.102864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5b4a5-1a5a-4f1e-ba3a-7ad614555c8b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:19.138854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297f3b8-56fa-4b69-b920-e541d66d33bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:19.139463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297f3b8-56fa-4b69-b920-e541d66d33bf None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:19.139750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297f3b8-56fa-4b69-b920-e541d66d33bf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:19.140026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297f3b8-56fa-4b69-b920-e541d66d33bf None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:19.140776 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297f3b8-56fa-4b69-b920-e541d66d33bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:19.142156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297f3b8-56fa-4b69-b920-e541d66d33bf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:19.157977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 485/980] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:25:18 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:25:19.206250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc24ed6-dfca-4d5b-ac76-1e28b97ef71d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:19.252397 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25635acd-a64c-499e-885c-161b9b663218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:19.253132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25635acd-a64c-499e-885c-161b9b663218 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:19.253309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25635acd-a64c-499e-885c-161b9b663218 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:19.253579 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25635acd-a64c-499e-885c-161b9b663218 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:19.254339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25635acd-a64c-499e-885c-161b9b663218 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:19.254653 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25635acd-a64c-499e-885c-161b9b663218 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:19.256201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25635acd-a64c-499e-885c-161b9b663218 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:19.276180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 496/981] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:25:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:19.278105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297f3b8-56fa-4b69-b920-e541d66d33bf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:19.387226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:19.398891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297f3b8-56fa-4b69-b920-e541d66d33bf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:19.433415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e9bce-4f65-4191-9c16-e4bb43a62090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:19.434034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e9bce-4f65-4191-9c16-e4bb43a62090 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:19.434319 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e9bce-4f65-4191-9c16-e4bb43a62090 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:19.434597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e9bce-4f65-4191-9c16-e4bb43a62090 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:19.435416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e9bce-4f65-4191-9c16-e4bb43a62090 None admin] RBAC: Adding query filter params (role.id=d1e0bfc9cbb14751b3daabc5e2de9813, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=8340fafe48af45c8b87a97aaac637bd5) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:19.435733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e9bce-4f65-4191-9c16-e4bb43a62090 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:19.437162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e9bce-4f65-4191-9c16-e4bb43a62090 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:19.451143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 486/982] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:25:19 2022] GET /identity/v3/role_assignments?user.id=8340fafe48af45c8b87a97aaac637bd5&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=d1e0bfc9cbb14751b3daabc5e2de9813 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:19.570647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af7fd5-d962-4703-8c78-753ae23bd924 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:19.571410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af7fd5-d962-4703-8c78-753ae23bd924 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:19.571410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af7fd5-d962-4703-8c78-753ae23bd924 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:19.572804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 497/983] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:19.578274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca939db1-9368-4cc5-8371-5b9d6dcf76f8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:19.578568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca939db1-9368-4cc5-8371-5b9d6dcf76f8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:19.578847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca939db1-9368-4cc5-8371-5b9d6dcf76f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:19.597363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ca939db1-9368-4cc5-8371-5b9d6dcf76f8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:19.598564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ca939db1-9368-4cc5-8371-5b9d6dcf76f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:19.670143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 487/984] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:19.693280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af7fd5-d962-4703-8c78-753ae23bd924 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:19.750363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:19.800706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af7fd5-d962-4703-8c78-753ae23bd924 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:19.836799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a7a90-09ad-4e10-b096-9b49783a9559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:19.837446 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a7a90-09ad-4e10-b096-9b49783a9559 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:19.837734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a7a90-09ad-4e10-b096-9b49783a9559 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:19.838013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a7a90-09ad-4e10-b096-9b49783a9559 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:19.838643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a7a90-09ad-4e10-b096-9b49783a9559 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:19.840067 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a7a90-09ad-4e10-b096-9b49783a9559 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-141a7a90-09ad-4e10-b096-9b49783a9559 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:19.849084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:19.850030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:19.851118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:19.855632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 498/985] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Aug 1 07:25:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:19.856760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca939db1-9368-4cc5-8371-5b9d6dcf76f8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:19.945708 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:19.966336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca939db1-9368-4cc5-8371-5b9d6dcf76f8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:20.002278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc5c3f-2f7d-4a08-b5b7-3b529cab06f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:20.002898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc5c3f-2f7d-4a08-b5b7-3b529cab06f9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:20.003209 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc5c3f-2f7d-4a08-b5b7-3b529cab06f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:20.003488 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc5c3f-2f7d-4a08-b5b7-3b529cab06f9 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:20.004181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc5c3f-2f7d-4a08-b5b7-3b529cab06f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:20.004462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc5c3f-2f7d-4a08-b5b7-3b529cab06f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:20.005818 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc5c3f-2f7d-4a08-b5b7-3b529cab06f9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:20.015761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 488/986] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Aug 1 07:25:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:20.016783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a7a90-09ad-4e10-b096-9b49783a9559 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:20.123246 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a7a90-09ad-4e10-b096-9b49783a9559 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:20.141179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:20.160448 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853601d9-c358-48a6-a1da-601a56a38c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:20.161071 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853601d9-c358-48a6-a1da-601a56a38c4a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:20.161353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853601d9-c358-48a6-a1da-601a56a38c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:20.161639 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853601d9-c358-48a6-a1da-601a56a38c4a None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:20.162328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853601d9-c358-48a6-a1da-601a56a38c4a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:20.163184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853601d9-c358-48a6-a1da-601a56a38c4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:20.164639 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853601d9-c358-48a6-a1da-601a56a38c4a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:20.174500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 499/987] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Aug 1 07:25:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:20.175569 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc5c3f-2f7d-4a08-b5b7-3b529cab06f9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:20.279808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc5c3f-2f7d-4a08-b5b7-3b529cab06f9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:20.314936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a0c33-5c7e-46ed-9321-ec2dbe935187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:20.315581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a0c33-5c7e-46ed-9321-ec2dbe935187 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:20.316254 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a0c33-5c7e-46ed-9321-ec2dbe935187 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:20.316541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a0c33-5c7e-46ed-9321-ec2dbe935187 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:20.317306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a0c33-5c7e-46ed-9321-ec2dbe935187 None admin] RBAC: Adding query filter params (interface=admin, service_id=a67d65cbf1794454857e439406d459fd, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:20.317616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a0c33-5c7e-46ed-9321-ec2dbe935187 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:20.319716 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a0c33-5c7e-46ed-9321-ec2dbe935187 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:20.329355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 489/988] 10.222.0.40 () {56 vars in 1249 bytes} [Mon Aug 1 07:25:20 2022] GET /identity/v3/endpoints?service_id=a67d65cbf1794454857e439406d459fd&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:20.330600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853601d9-c358-48a6-a1da-601a56a38c4a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:20.432039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853601d9-c358-48a6-a1da-601a56a38c4a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:20.466015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55903e30-1481-4cfe-8af9-f3bdf90a159d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:20.466637 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55903e30-1481-4cfe-8af9-f3bdf90a159d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:20.466926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55903e30-1481-4cfe-8af9-f3bdf90a159d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:20.467240 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55903e30-1481-4cfe-8af9-f3bdf90a159d None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:20.467893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55903e30-1481-4cfe-8af9-f3bdf90a159d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:20.469262 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55903e30-1481-4cfe-8af9-f3bdf90a159d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:20.475986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 500/989] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Aug 1 07:25:20 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:25:21.191905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdeeb21-239f-4a69-9b31-8091869b2ee4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:21.192304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdeeb21-239f-4a69-9b31-8091869b2ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:21.192482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdeeb21-239f-4a69-9b31-8091869b2ee4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:21.195111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 490/990] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:21.199488 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a524d1d-a1fa-4604-99f0-851195dd2264 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:21.199915 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a524d1d-a1fa-4604-99f0-851195dd2264 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:21.200103 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a524d1d-a1fa-4604-99f0-851195dd2264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:21.219464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9a524d1d-a1fa-4604-99f0-851195dd2264 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:21.220596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9a524d1d-a1fa-4604-99f0-851195dd2264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:21.264333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:21.291919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 501/991] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:21.314786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdeeb21-239f-4a69-9b31-8091869b2ee4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:21.332648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:21.422006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdeeb21-239f-4a69-9b31-8091869b2ee4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:21.456345 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11162931-3c4d-49fd-8607-c782b04b6517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:21.457015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11162931-3c4d-49fd-8607-c782b04b6517 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:21.457304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11162931-3c4d-49fd-8607-c782b04b6517 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:21.457583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11162931-3c4d-49fd-8607-c782b04b6517 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:21.458205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11162931-3c4d-49fd-8607-c782b04b6517 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:21.459667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11162931-3c4d-49fd-8607-c782b04b6517 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-11162931-3c4d-49fd-8607-c782b04b6517 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:21.468723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:21.469749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:25:21.470624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:21.474814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 491/992] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:21.476799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a524d1d-a1fa-4604-99f0-851195dd2264 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:21.489284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a4e5c-a81d-4d7a-bf58-c26e05aa5db0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:21.489605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a4e5c-a81d-4d7a-bf58-c26e05aa5db0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:21.489889 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a4e5c-a81d-4d7a-bf58-c26e05aa5db0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:21.492227 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 492/993] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:25:21.496916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d45ee-1214-4e20-8178-4cb0b6d8ee74 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:21.497221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d45ee-1214-4e20-8178-4cb0b6d8ee74 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:21.497491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d45ee-1214-4e20-8178-4cb0b6d8ee74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:21.516223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4a0d45ee-1214-4e20-8178-4cb0b6d8ee74 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:21.517443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4a0d45ee-1214-4e20-8178-4cb0b6d8ee74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:21.579208 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a524d1d-a1fa-4604-99f0-851195dd2264 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:21.589679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 493/994] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:21.612182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d45ee-1214-4e20-8178-4cb0b6d8ee74 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:21.617012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799c9ee-1841-41eb-9efb-914f2529b0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:21.617640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799c9ee-1841-41eb-9efb-914f2529b0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:21.617931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799c9ee-1841-41eb-9efb-914f2529b0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:21.618212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799c9ee-1841-41eb-9efb-914f2529b0a2 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:21.618923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799c9ee-1841-41eb-9efb-914f2529b0a2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:21.619266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799c9ee-1841-41eb-9efb-914f2529b0a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:21.620654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799c9ee-1841-41eb-9efb-914f2529b0a2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:21.622687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:21.627308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 502/995] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:25:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:21.634201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799c9ee-1841-41eb-9efb-914f2529b0a2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:21.676871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:21.727626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d45ee-1214-4e20-8178-4cb0b6d8ee74 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:21.739606 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799c9ee-1841-41eb-9efb-914f2529b0a2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:21.764362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ce480-86fa-492a-bd4c-7e7eabd55489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:21.765006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ce480-86fa-492a-bd4c-7e7eabd55489 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:21.765410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ce480-86fa-492a-bd4c-7e7eabd55489 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:21.765692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ce480-86fa-492a-bd4c-7e7eabd55489 None admin] PATH_INFO: `/v3/services/image` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:21.766443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ce480-86fa-492a-bd4c-7e7eabd55489 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:21.767844 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ce480-86fa-492a-bd4c-7e7eabd55489 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:21.774182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a05ec9-b345-43db-bc8e-7dc8f4d6bfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:21.774800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a05ec9-b345-43db-bc8e-7dc8f4d6bfbf None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:21.775110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a05ec9-b345-43db-bc8e-7dc8f4d6bfbf None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:21.775396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a05ec9-b345-43db-bc8e-7dc8f4d6bfbf None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:21.776068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a05ec9-b345-43db-bc8e-7dc8f4d6bfbf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:21.776381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a05ec9-b345-43db-bc8e-7dc8f4d6bfbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5e0ce480-86fa-492a-bd4c-7e7eabd55489 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:21.777565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:21.778499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:25:21.779366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:21.780091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a05ec9-b345-43db-bc8e-7dc8f4d6bfbf None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:21.783657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 494/996] 10.222.0.40 () {56 vars in 1105 bytes} [Mon Aug 1 07:25:21 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:21.784394 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ce480-86fa-492a-bd4c-7e7eabd55489 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:21.789749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 503/997] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:25:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:21.790573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a05ec9-b345-43db-bc8e-7dc8f4d6bfbf None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:21.862205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:21.893346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a05ec9-b345-43db-bc8e-7dc8f4d6bfbf None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:21.893952 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ce480-86fa-492a-bd4c-7e7eabd55489 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:21.894992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:21.931407 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c31970-966f-462f-8da3-6ee9b290f612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:21.931663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8e3ee-66a3-4520-8db4-f9d0517e7ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:21.932029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c31970-966f-462f-8da3-6ee9b290f612 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:21.932136 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8e3ee-66a3-4520-8db4-f9d0517e7ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:21.932331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c31970-966f-462f-8da3-6ee9b290f612 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:21.932434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8e3ee-66a3-4520-8db4-f9d0517e7ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:21.932604 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c31970-966f-462f-8da3-6ee9b290f612 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:21.932709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8e3ee-66a3-4520-8db4-f9d0517e7ec2 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:21.933275 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8e3ee-66a3-4520-8db4-f9d0517e7ec2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:21.933381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c31970-966f-462f-8da3-6ee9b290f612 None admin] RBAC: Adding query filter params (interface=public, service_id=a80d4123c57d4d04ac4057e58717348f, region_id=RegionOne) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:21.933578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8e3ee-66a3-4520-8db4-f9d0517e7ec2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:21.933675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c31970-966f-462f-8da3-6ee9b290f612 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:21.934943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8e3ee-66a3-4520-8db4-f9d0517e7ec2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:21.935051 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c31970-966f-462f-8da3-6ee9b290f612 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:21.940082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 495/998] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:21.945003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 504/999] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Aug 1 07:25:21 2022] GET /identity/v3/endpoints?service_id=a80d4123c57d4d04ac4057e58717348f&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:21.946130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c31970-966f-462f-8da3-6ee9b290f612 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:21.946267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8e3ee-66a3-4520-8db4-f9d0517e7ec2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:22.062001 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c31970-966f-462f-8da3-6ee9b290f612 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:22.064794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8e3ee-66a3-4520-8db4-f9d0517e7ec2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:22.099920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ec6a3-382e-43e6-8f0c-6b76265e4589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:22.101337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ec6a3-382e-43e6-8f0c-6b76265e4589 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:22.101737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ec6a3-382e-43e6-8f0c-6b76265e4589 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:22.102020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ec6a3-382e-43e6-8f0c-6b76265e4589 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:22.102410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3770f-026d-445f-bc93-0d6838b4e092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:22.102821 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ec6a3-382e-43e6-8f0c-6b76265e4589 None admin] RBAC: Adding query filter params (type=image) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:22.103059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3770f-026d-445f-bc93-0d6838b4e092 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:22.103241 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ec6a3-382e-43e6-8f0c-6b76265e4589 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:22.103375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3770f-026d-445f-bc93-0d6838b4e092 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:22.103641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3770f-026d-445f-bc93-0d6838b4e092 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:22.104293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3770f-026d-445f-bc93-0d6838b4e092 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:22.104901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ec6a3-382e-43e6-8f0c-6b76265e4589 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:22.105642 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3770f-026d-445f-bc93-0d6838b4e092 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:22.107724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:22.111111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 505/1000] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:22.113726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 496/1001] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Aug 1 07:25:21 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:25:22.502468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d884f-e7aa-48a6-9d3a-9efea3c9d785 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:22.503012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d884f-e7aa-48a6-9d3a-9efea3c9d785 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:22.503012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d884f-e7aa-48a6-9d3a-9efea3c9d785 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:22.504548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 506/1002] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:22.509966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0a152-b4c3-4eda-a56c-246a146dec44 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:22.510353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0a152-b4c3-4eda-a56c-246a146dec44 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:22.510528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0a152-b4c3-4eda-a56c-246a146dec44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:22.529276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-53e0a152-b4c3-4eda-a56c-246a146dec44 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:22.530484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-53e0a152-b4c3-4eda-a56c-246a146dec44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:22.601449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 497/1003] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:22.623448 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d884f-e7aa-48a6-9d3a-9efea3c9d785 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:22.655221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:22.728287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d884f-e7aa-48a6-9d3a-9efea3c9d785 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:22.763456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee07885-da3e-4c6f-aedc-fffe4a719e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:22.764030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee07885-da3e-4c6f-aedc-fffe4a719e18 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:22.764310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee07885-da3e-4c6f-aedc-fffe4a719e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:22.764583 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee07885-da3e-4c6f-aedc-fffe4a719e18 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:22.765206 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee07885-da3e-4c6f-aedc-fffe4a719e18 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:22.766560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee07885-da3e-4c6f-aedc-fffe4a719e18 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9ee07885-da3e-4c6f-aedc-fffe4a719e18 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:22.775731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:22.776725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 07:25:22.777588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:22.779096 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 507/1004] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Aug 1 07:25:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:22.782973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0a152-b4c3-4eda-a56c-246a146dec44 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:22.885358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0a152-b4c3-4eda-a56c-246a146dec44 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:22.914578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:22.921393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e4dda-b669-4312-b8a5-7e39fe626b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:22.922245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e4dda-b669-4312-b8a5-7e39fe626b17 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:22.922711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e4dda-b669-4312-b8a5-7e39fe626b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:22.923233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e4dda-b669-4312-b8a5-7e39fe626b17 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:22.924083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e4dda-b669-4312-b8a5-7e39fe626b17 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:22.924587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e4dda-b669-4312-b8a5-7e39fe626b17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:22.926135 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e4dda-b669-4312-b8a5-7e39fe626b17 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:22.932699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 498/1005] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Aug 1 07:25:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:22.938599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee07885-da3e-4c6f-aedc-fffe4a719e18 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:23.044657 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:23.052220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee07885-da3e-4c6f-aedc-fffe4a719e18 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:23.087512 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71d005-651e-4fdb-90d4-4d59920e44cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:23.088130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71d005-651e-4fdb-90d4-4d59920e44cf None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:23.088413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71d005-651e-4fdb-90d4-4d59920e44cf None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:23.088689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71d005-651e-4fdb-90d4-4d59920e44cf None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:23.089359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71d005-651e-4fdb-90d4-4d59920e44cf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:23.089666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71d005-651e-4fdb-90d4-4d59920e44cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:23.091016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71d005-651e-4fdb-90d4-4d59920e44cf None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:23.096149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 508/1006] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Aug 1 07:25:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:23.102465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e4dda-b669-4312-b8a5-7e39fe626b17 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:23.204315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e4dda-b669-4312-b8a5-7e39fe626b17 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:23.244711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec63f49-5589-4195-af18-613edd2bfc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:23.245341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec63f49-5589-4195-af18-613edd2bfc62 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:23.245630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec63f49-5589-4195-af18-613edd2bfc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:23.245911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec63f49-5589-4195-af18-613edd2bfc62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:23.246628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec63f49-5589-4195-af18-613edd2bfc62 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:23.248072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec63f49-5589-4195-af18-613edd2bfc62 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:23.266764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 499/1007] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:25:23 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:25:24.114190 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef8a57-c7fb-4b7e-af96-fccb6754d42c None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:24.114735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef8a57-c7fb-4b7e-af96-fccb6754d42c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:24.114735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef8a57-c7fb-4b7e-af96-fccb6754d42c None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:24.119108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 509/1008] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:24.121941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9659c8d-9138-4d30-b732-6c0c38542c7f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:24.122335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9659c8d-9138-4d30-b732-6c0c38542c7f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:24.122509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9659c8d-9138-4d30-b732-6c0c38542c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:24.142153 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a9659c8d-9138-4d30-b732-6c0c38542c7f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:24.143479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a9659c8d-9138-4d30-b732-6c0c38542c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:24.161758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcda30-41ce-472f-bd4b-349ba9175ff3 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:24.162054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcda30-41ce-472f-bd4b-349ba9175ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:24.162341 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcda30-41ce-472f-bd4b-349ba9175ff3 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:24.164498 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 510/1009] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:24.169376 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a400a-e6f9-4c6d-a0a1-239e715790df None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:24.169868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a400a-e6f9-4c6d-a0a1-239e715790df None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:24.170485 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a400a-e6f9-4c6d-a0a1-239e715790df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:24.194855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:24.195661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-fe3a400a-e6f9-4c6d-a0a1-239e715790df None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:24.197468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-fe3a400a-e6f9-4c6d-a0a1-239e715790df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:24.222471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 500/1010] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:24 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:24.246220 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9659c8d-9138-4d30-b732-6c0c38542c7f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:24.271547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 511/1011] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:24 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:24.293586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a400a-e6f9-4c6d-a0a1-239e715790df None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:24.352982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9659c8d-9138-4d30-b732-6c0c38542c7f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:24.384042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:24.389814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431fde5-03a5-43ab-abc8-9fdb10e7a093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:24.390454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431fde5-03a5-43ab-abc8-9fdb10e7a093 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:24.390736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431fde5-03a5-43ab-abc8-9fdb10e7a093 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:24.391013 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431fde5-03a5-43ab-abc8-9fdb10e7a093 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:24.391760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431fde5-03a5-43ab-abc8-9fdb10e7a093 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:24.393170 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431fde5-03a5-43ab-abc8-9fdb10e7a093 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:24.399960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a400a-e6f9-4c6d-a0a1-239e715790df None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:24.414948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 501/1012] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:25:24 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:25:24.446837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff82892-945d-47fd-b682-cee0ea3a24cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:24.447527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff82892-945d-47fd-b682-cee0ea3a24cd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:24.447810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff82892-945d-47fd-b682-cee0ea3a24cd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:24.448082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff82892-945d-47fd-b682-cee0ea3a24cd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:24.448719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff82892-945d-47fd-b682-cee0ea3a24cd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:24.450245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff82892-945d-47fd-b682-cee0ea3a24cd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-eff82892-945d-47fd-b682-cee0ea3a24cd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:24.460354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:24.461404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 07:25:24.462252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:24.466923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 512/1013] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:24.468284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431fde5-03a5-43ab-abc8-9fdb10e7a093 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:24.516305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:24.582867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431fde5-03a5-43ab-abc8-9fdb10e7a093 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:24.617805 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de11f4-2977-49c5-8b7d-984954d54c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:24.618434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de11f4-2977-49c5-8b7d-984954d54c63 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:24.618725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de11f4-2977-49c5-8b7d-984954d54c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:24.619002 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de11f4-2977-49c5-8b7d-984954d54c63 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:24.619732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de11f4-2977-49c5-8b7d-984954d54c63 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:24.620041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de11f4-2977-49c5-8b7d-984954d54c63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:24.621427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de11f4-2977-49c5-8b7d-984954d54c63 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:24.626783 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 502/1014] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:25:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:24.633023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff82892-945d-47fd-b682-cee0ea3a24cd None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:24.736610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff82892-945d-47fd-b682-cee0ea3a24cd None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:24.771156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498b0d5-fb8d-48e5-8591-8d131e698e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:24.772391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498b0d5-fb8d-48e5-8591-8d131e698e80 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:24.772752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498b0d5-fb8d-48e5-8591-8d131e698e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:24.772998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498b0d5-fb8d-48e5-8591-8d131e698e80 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:24.773679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498b0d5-fb8d-48e5-8591-8d131e698e80 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:24.773986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498b0d5-fb8d-48e5-8591-8d131e698e80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:24.775371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498b0d5-fb8d-48e5-8591-8d131e698e80 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:24.780419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 513/1015] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Aug 1 07:25:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:24.785870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de11f4-2977-49c5-8b7d-984954d54c63 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:24.888100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de11f4-2977-49c5-8b7d-984954d54c63 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:24.890314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:24.924713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416afd2-9738-44a5-881d-4ddc93b28246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:24.925342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416afd2-9738-44a5-881d-4ddc93b28246 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:24.925630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416afd2-9738-44a5-881d-4ddc93b28246 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:24.925907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416afd2-9738-44a5-881d-4ddc93b28246 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:24.926624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416afd2-9738-44a5-881d-4ddc93b28246 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:24.928156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416afd2-9738-44a5-881d-4ddc93b28246 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:24.959107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 503/1016] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:25:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:25:25.332559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202b14-694d-4a0d-9d3f-eb8c7de95a63 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:25.333079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202b14-694d-4a0d-9d3f-eb8c7de95a63 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:25.333158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202b14-694d-4a0d-9d3f-eb8c7de95a63 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:25.334728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 514/1017] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:25.339947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4330fcd-de20-4d8a-8acc-24fa3a86e168 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:25.340342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4330fcd-de20-4d8a-8acc-24fa3a86e168 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:25.340513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4330fcd-de20-4d8a-8acc-24fa3a86e168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:25.359131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e4330fcd-de20-4d8a-8acc-24fa3a86e168 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:25.360334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e4330fcd-de20-4d8a-8acc-24fa3a86e168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:25.434131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 504/1018] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:25.457377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3202b14-694d-4a0d-9d3f-eb8c7de95a63 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:25.473658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:25.566374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3202b14-694d-4a0d-9d3f-eb8c7de95a63 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:25.605115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1313ca-56c8-4bc6-a7a9-5821d03f52ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:25.605983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1313ca-56c8-4bc6-a7a9-5821d03f52ed None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:25.606431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1313ca-56c8-4bc6-a7a9-5821d03f52ed None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:25.606897 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1313ca-56c8-4bc6-a7a9-5821d03f52ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:25.608369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1313ca-56c8-4bc6-a7a9-5821d03f52ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:25.610006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1313ca-56c8-4bc6-a7a9-5821d03f52ed None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:25.612219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 515/1019] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:25.617823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4330fcd-de20-4d8a-8acc-24fa3a86e168 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:25.746237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:25.774467 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4330fcd-de20-4d8a-8acc-24fa3a86e168 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:25.831683 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c0eb0-3901-455e-b4ef-8183bfcf172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:25.832726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c0eb0-3901-455e-b4ef-8183bfcf172d None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:25.833221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c0eb0-3901-455e-b4ef-8183bfcf172d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:25.833726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c0eb0-3901-455e-b4ef-8183bfcf172d None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:25.835550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c0eb0-3901-455e-b4ef-8183bfcf172d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:25.838235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c0eb0-3901-455e-b4ef-8183bfcf172d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:25.890077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 505/1020] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:25:25 2022] POST /identity/v3/projects => generated 311 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:25:26.555005 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128c7d8-1d06-411c-a257-944ce62a9287 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:26.555557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128c7d8-1d06-411c-a257-944ce62a9287 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:26.555639 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128c7d8-1d06-411c-a257-944ce62a9287 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:26.556974 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 516/1021] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:26.562165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba541063-f47e-4ace-992b-dafed40c5f0c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:26.562661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba541063-f47e-4ace-992b-dafed40c5f0c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:26.562742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba541063-f47e-4ace-992b-dafed40c5f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:26.581348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ba541063-f47e-4ace-992b-dafed40c5f0c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:26.582524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ba541063-f47e-4ace-992b-dafed40c5f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:26.651328 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 506/1022] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:26.673834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128c7d8-1d06-411c-a257-944ce62a9287 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:26.757818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:26.779728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128c7d8-1d06-411c-a257-944ce62a9287 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:26.812997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0535d9b-bfba-4008-9067-3363abca3d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:26.813570 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0535d9b-bfba-4008-9067-3363abca3d33 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:26.813855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0535d9b-bfba-4008-9067-3363abca3d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:26.814131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0535d9b-bfba-4008-9067-3363abca3d33 None admin] PATH_INFO: `/v3/services/image` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:26.814752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0535d9b-bfba-4008-9067-3363abca3d33 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:26.816153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0535d9b-bfba-4008-9067-3363abca3d33 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-f0535d9b-bfba-4008-9067-3363abca3d33 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:26.825659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:26.826660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:25:26.827560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:26.831384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 517/1023] 10.222.0.40 () {56 vars in 1105 bytes} [Mon Aug 1 07:25:26 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:26.832882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba541063-f47e-4ace-992b-dafed40c5f0c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:26.900447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:26.937160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba541063-f47e-4ace-992b-dafed40c5f0c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:26.974410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f28bf-145d-4165-a6d8-e08b994d64eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:26.975353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f28bf-145d-4165-a6d8-e08b994d64eb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:26.975646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f28bf-145d-4165-a6d8-e08b994d64eb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:26.975929 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f28bf-145d-4165-a6d8-e08b994d64eb None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:26.976616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f28bf-145d-4165-a6d8-e08b994d64eb None admin] RBAC: Adding query filter params (name=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:26.976921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f28bf-145d-4165-a6d8-e08b994d64eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:26.978293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f28bf-145d-4165-a6d8-e08b994d64eb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:26.990555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 507/1024] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:26.996541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0535d9b-bfba-4008-9067-3363abca3d33 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:27.060712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504b4c3-3fc3-408b-a042-c3f28b0b3cd3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:27.061652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504b4c3-3fc3-408b-a042-c3f28b0b3cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:27.062348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504b4c3-3fc3-408b-a042-c3f28b0b3cd3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:27.065212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 508/1025] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:25:27.070550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2f1e7-2aad-4bd0-8f1c-90823c21d54f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:27.070836 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2f1e7-2aad-4bd0-8f1c-90823c21d54f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:27.071134 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2f1e7-2aad-4bd0-8f1c-90823c21d54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:27.089120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4af2f1e7-2aad-4bd0-8f1c-90823c21d54f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:27.090314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4af2f1e7-2aad-4bd0-8f1c-90823c21d54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:27.110179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0535d9b-bfba-4008-9067-3363abca3d33 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:27.146152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e34d45-b724-4fa9-934f-22514b8658a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:27.146824 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e34d45-b724-4fa9-934f-22514b8658a1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:27.147137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e34d45-b724-4fa9-934f-22514b8658a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:27.147425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e34d45-b724-4fa9-934f-22514b8658a1 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:27.148097 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e34d45-b724-4fa9-934f-22514b8658a1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:27.148482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e34d45-b724-4fa9-934f-22514b8658a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:27.149865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e34d45-b724-4fa9-934f-22514b8658a1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:27.155049 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 518/1026] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:27.160765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e34d45-b724-4fa9-934f-22514b8658a1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:27.162757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 509/1027] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:27.176216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:27.185547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2f1e7-2aad-4bd0-8f1c-90823c21d54f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:27.250540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:27.275276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e34d45-b724-4fa9-934f-22514b8658a1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:27.296731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2f1e7-2aad-4bd0-8f1c-90823c21d54f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:27.309450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20e278-691d-4ad5-87f6-501b1cda283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:27.310076 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20e278-691d-4ad5-87f6-501b1cda283c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:27.310363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20e278-691d-4ad5-87f6-501b1cda283c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:27.310642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20e278-691d-4ad5-87f6-501b1cda283c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:27.311427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20e278-691d-4ad5-87f6-501b1cda283c None admin] RBAC: Adding query filter params (interface=public, service_id=888f3758656e41e2be988198c7ed44bf, region_id=RegionOne) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:27.311746 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20e278-691d-4ad5-87f6-501b1cda283c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:27.313115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20e278-691d-4ad5-87f6-501b1cda283c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:27.318502 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 519/1028] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Aug 1 07:25:27 2022] GET /identity/v3/endpoints?service_id=888f3758656e41e2be988198c7ed44bf&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:27.324515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20e278-691d-4ad5-87f6-501b1cda283c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:27.338426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a7a1d-a1db-4147-9c53-d0ea4301b05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:27.339096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a7a1d-a1db-4147-9c53-d0ea4301b05e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:27.339385 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a7a1d-a1db-4147-9c53-d0ea4301b05e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:27.339694 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a7a1d-a1db-4147-9c53-d0ea4301b05e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:27.350278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a7a1d-a1db-4147-9c53-d0ea4301b05e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-289a7a1d-a1db-4147-9c53-d0ea4301b05e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:27.355199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:27.356178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:27.360159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 510/1029] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Aug 1 07:25:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:27.362282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a7a1d-a1db-4147-9c53-d0ea4301b05e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:27.409873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:27.442868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20e278-691d-4ad5-87f6-501b1cda283c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:27.471847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a7a1d-a1db-4147-9c53-d0ea4301b05e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:27.480608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fa852-5bb0-49a5-b1d5-33f960235bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:27.481320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fa852-5bb0-49a5-b1d5-33f960235bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:27.481610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fa852-5bb0-49a5-b1d5-33f960235bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:27.481894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fa852-5bb0-49a5-b1d5-33f960235bd4 None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:27.482550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fa852-5bb0-49a5-b1d5-33f960235bd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:27.483964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fa852-5bb0-49a5-b1d5-33f960235bd4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:27.490622 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 520/1030] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Aug 1 07:25:27 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:25:27.508140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:27.523304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf2bdd-83c2-4fc3-8127-09e43ad0b108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:27.523944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf2bdd-83c2-4fc3-8127-09e43ad0b108 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:27.524229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf2bdd-83c2-4fc3-8127-09e43ad0b108 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:27.524506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf2bdd-83c2-4fc3-8127-09e43ad0b108 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:27.525238 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf2bdd-83c2-4fc3-8127-09e43ad0b108 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:27.525545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf2bdd-83c2-4fc3-8127-09e43ad0b108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:27.526958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf2bdd-83c2-4fc3-8127-09e43ad0b108 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:27.536752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 511/1031] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Aug 1 07:25:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:27.542617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fa852-5bb0-49a5-b1d5-33f960235bd4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:27.658294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fa852-5bb0-49a5-b1d5-33f960235bd4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:27.694033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f66856-76ec-4e4a-acb4-e36fca57af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:27.694683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f66856-76ec-4e4a-acb4-e36fca57af6d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:27.694970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f66856-76ec-4e4a-acb4-e36fca57af6d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:27.695285 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f66856-76ec-4e4a-acb4-e36fca57af6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:27.698376 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:27.707350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f66856-76ec-4e4a-acb4-e36fca57af6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:27.708930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f66856-76ec-4e4a-acb4-e36fca57af6d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-08f66856-76ec-4e4a-acb4-e36fca57af6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:27.724473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:27.725543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:27.726987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:27.726987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:27.726987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:27.726987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:27.726987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 521/1032] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:27.731423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf2bdd-83c2-4fc3-8127-09e43ad0b108 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:27.832437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf2bdd-83c2-4fc3-8127-09e43ad0b108 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:27.866614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad53b8c-0e78-4cf8-9727-84ee913e7670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:27.867263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad53b8c-0e78-4cf8-9727-84ee913e7670 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:27.867545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad53b8c-0e78-4cf8-9727-84ee913e7670 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:27.867816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad53b8c-0e78-4cf8-9727-84ee913e7670 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:27.868501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad53b8c-0e78-4cf8-9727-84ee913e7670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:27.868804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad53b8c-0e78-4cf8-9727-84ee913e7670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:27.870148 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad53b8c-0e78-4cf8-9727-84ee913e7670 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:27.882700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 512/1033] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:27.888298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f66856-76ec-4e4a-acb4-e36fca57af6d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:27.961945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e0c03-258d-462b-b115-a9ee9481a834 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:27.961945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e0c03-258d-462b-b115-a9ee9481a834 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:27.961945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e0c03-258d-462b-b115-a9ee9481a834 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:27.961945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 513/1034] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:25:27.966719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cc572-fca6-4f85-b8aa-9cbd00a95fdd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:27.967004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cc572-fca6-4f85-b8aa-9cbd00a95fdd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:27.967328 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cc572-fca6-4f85-b8aa-9cbd00a95fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:27.985039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9c2cc572-fca6-4f85-b8aa-9cbd00a95fdd None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:27.986220 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9c2cc572-fca6-4f85-b8aa-9cbd00a95fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:28.002077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f66856-76ec-4e4a-acb4-e36fca57af6d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:28.028957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:28.043274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc1202-0105-4978-bdd2-788bb218bd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:28.043921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc1202-0105-4978-bdd2-788bb218bd84 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:28.044208 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc1202-0105-4978-bdd2-788bb218bd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:28.044491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc1202-0105-4978-bdd2-788bb218bd84 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:28.053677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc1202-0105-4978-bdd2-788bb218bd84 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:28.055312 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc1202-0105-4978-bdd2-788bb218bd84 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:28.061064 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 514/1035] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-d1fc1202-0105-4978-bdd2-788bb218bd84 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:25:28.067037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:28.068046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:28.069069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:28.069069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:28.069069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:25:28.069069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:25:28.069069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:25:28.069069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:25:28.069069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:25:28.069069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:25:28.069069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:25:28.069069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:28.071346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 522/1036] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Aug 1 07:25:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:28.074475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cc572-fca6-4f85-b8aa-9cbd00a95fdd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:28.087866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc1202-0105-4978-bdd2-788bb218bd84 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:28.140745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:28.182224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cc572-fca6-4f85-b8aa-9cbd00a95fdd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:28.196714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc1202-0105-4978-bdd2-788bb218bd84 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:28.216765 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88224af-e53f-47b9-b2c8-844afc786c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:28.217483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88224af-e53f-47b9-b2c8-844afc786c3f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:28.217777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88224af-e53f-47b9-b2c8-844afc786c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:28.218040 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88224af-e53f-47b9-b2c8-844afc786c3f None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:28.218835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88224af-e53f-47b9-b2c8-844afc786c3f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:28.219168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88224af-e53f-47b9-b2c8-844afc786c3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:28.220596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88224af-e53f-47b9-b2c8-844afc786c3f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:28.233365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7ff38-9b71-4785-8e5d-28cd3a1d9755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:28.234258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7ff38-9b71-4785-8e5d-28cd3a1d9755 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:28.234841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7ff38-9b71-4785-8e5d-28cd3a1d9755 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:28.235156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7ff38-9b71-4785-8e5d-28cd3a1d9755 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:28.236325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7ff38-9b71-4785-8e5d-28cd3a1d9755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:28.237760 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7ff38-9b71-4785-8e5d-28cd3a1d9755 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:28.238302 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 515/1037] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Aug 1 07:25:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:28.244648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 523/1038] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:28.244801 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88224af-e53f-47b9-b2c8-844afc786c3f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:28.245388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7ff38-9b71-4785-8e5d-28cd3a1d9755 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:28.274798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:28.348917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88224af-e53f-47b9-b2c8-844afc786c3f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:28.356560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7ff38-9b71-4785-8e5d-28cd3a1d9755 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:28.374875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:28.384802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38aae1-55c3-4ff8-b12a-02e50e9efbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:28.385420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38aae1-55c3-4ff8-b12a-02e50e9efbcd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:28.385704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38aae1-55c3-4ff8-b12a-02e50e9efbcd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:28.385982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38aae1-55c3-4ff8-b12a-02e50e9efbcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:28.397907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c025c-92c9-480e-9fe7-e879c9bbea5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:28.398525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c025c-92c9-480e-9fe7-e879c9bbea5d None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:28.398810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c025c-92c9-480e-9fe7-e879c9bbea5d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:28.399103 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c025c-92c9-480e-9fe7-e879c9bbea5d None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:28.399783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c025c-92c9-480e-9fe7-e879c9bbea5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:28.401301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c025c-92c9-480e-9fe7-e879c9bbea5d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:28.425435 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 524/1039] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:25:28 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:25:28.463774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38aae1-55c3-4ff8-b12a-02e50e9efbcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:28.465241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38aae1-55c3-4ff8-b12a-02e50e9efbcd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:28.498086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:28.574597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 516/1040] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:28 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 07:25:28.580577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c025c-92c9-480e-9fe7-e879c9bbea5d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:28.690060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c025c-92c9-480e-9fe7-e879c9bbea5d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:28.724997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7aad6a-b94f-49cc-9f2b-5505019b6244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:28.726022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aad6a-b94f-49cc-9f2b-5505019b6244 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:28.726513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aad6a-b94f-49cc-9f2b-5505019b6244 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:28.726981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aad6a-b94f-49cc-9f2b-5505019b6244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:28.738250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7aad6a-b94f-49cc-9f2b-5505019b6244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:28.739934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7aad6a-b94f-49cc-9f2b-5505019b6244 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-6d7aad6a-b94f-49cc-9f2b-5505019b6244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:28.755039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:28.756713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:28.757668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:28.757668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:28.757668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:28.757668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:28.762581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 525/1041] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:28.764012 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38aae1-55c3-4ff8-b12a-02e50e9efbcd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:28.863037 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38aae1-55c3-4ff8-b12a-02e50e9efbcd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:28.893799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:28.900188 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957af00-ccaf-43f0-9088-484372828fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:28.900809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957af00-ccaf-43f0-9088-484372828fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:28.901093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957af00-ccaf-43f0-9088-484372828fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:28.901370 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957af00-ccaf-43f0-9088-484372828fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:28.902052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957af00-ccaf-43f0-9088-484372828fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:28.902360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957af00-ccaf-43f0-9088-484372828fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:28.903755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957af00-ccaf-43f0-9088-484372828fe2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:28.920214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 517/1042] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:28.921432 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7aad6a-b94f-49cc-9f2b-5505019b6244 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:28.952910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:29.041372 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7aad6a-b94f-49cc-9f2b-5505019b6244 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:29.077660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a049ccb-5902-4a2f-90b2-3bb025eb6cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:29.078311 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a049ccb-5902-4a2f-90b2-3bb025eb6cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:29.078593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a049ccb-5902-4a2f-90b2-3bb025eb6cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:29.078871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a049ccb-5902-4a2f-90b2-3bb025eb6cd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:29.086794 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a049ccb-5902-4a2f-90b2-3bb025eb6cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:29.088288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a049ccb-5902-4a2f-90b2-3bb025eb6cd0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-0a049ccb-5902-4a2f-90b2-3bb025eb6cd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:25:29.100362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:29.101310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:29.102346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:25:29.102346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:25:29.102346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:25:29.102346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:25:29.102346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:25:29.102346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:25:29.102346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:29.102346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:29.102346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 526/1043] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:29.107698 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957af00-ccaf-43f0-9088-484372828fe2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:29.215798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957af00-ccaf-43f0-9088-484372828fe2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:29.249646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5365b2-3b39-4772-bbcd-79d92d513624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:29.250268 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5365b2-3b39-4772-bbcd-79d92d513624 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:29.250553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5365b2-3b39-4772-bbcd-79d92d513624 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:29.250832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5365b2-3b39-4772-bbcd-79d92d513624 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:29.251606 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5365b2-3b39-4772-bbcd-79d92d513624 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:29.251919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5365b2-3b39-4772-bbcd-79d92d513624 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:29.253337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5365b2-3b39-4772-bbcd-79d92d513624 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:29.266898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 518/1044] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:25:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:29.273299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a049ccb-5902-4a2f-90b2-3bb025eb6cd0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:29.360805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:29.378445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a049ccb-5902-4a2f-90b2-3bb025eb6cd0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:29.413186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61c1b8-3f62-49bb-8da9-93ef1ff413d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:29.413810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61c1b8-3f62-49bb-8da9-93ef1ff413d2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:29.414098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61c1b8-3f62-49bb-8da9-93ef1ff413d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:29.414379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61c1b8-3f62-49bb-8da9-93ef1ff413d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:29.415197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61c1b8-3f62-49bb-8da9-93ef1ff413d2 None admin] RBAC: Adding query filter params (role.id=d1e0bfc9cbb14751b3daabc5e2de9813, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=abdcd6d09fee45c696449641ea36eae3) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:29.415518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61c1b8-3f62-49bb-8da9-93ef1ff413d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:29.416948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61c1b8-3f62-49bb-8da9-93ef1ff413d2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:29.431094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 527/1045] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:25:29 2022] GET /identity/v3/role_assignments?user.id=abdcd6d09fee45c696449641ea36eae3&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=d1e0bfc9cbb14751b3daabc5e2de9813 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:29.520542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10291ab-8f54-475a-a2e4-76cad1b2f79d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:29.520894 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10291ab-8f54-475a-a2e4-76cad1b2f79d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:29.521131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10291ab-8f54-475a-a2e4-76cad1b2f79d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:29.522677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 519/1046] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:29.528021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671470c-5235-4538-b484-2969d7a29013 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:29.528309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671470c-5235-4538-b484-2969d7a29013 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:29.528586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671470c-5235-4538-b484-2969d7a29013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:29.547732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0671470c-5235-4538-b484-2969d7a29013 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:29.548960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0671470c-5235-4538-b484-2969d7a29013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:29.618474 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 528/1047] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:29.641678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10291ab-8f54-475a-a2e4-76cad1b2f79d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:29.697365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:29.746203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10291ab-8f54-475a-a2e4-76cad1b2f79d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:29.779511 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f93dd-6185-4a81-bffe-93ed1e1a5efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:29.780267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f93dd-6185-4a81-bffe-93ed1e1a5efd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:29.780492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f93dd-6185-4a81-bffe-93ed1e1a5efd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:29.780692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f93dd-6185-4a81-bffe-93ed1e1a5efd None admin] PATH_INFO: `/v3/services/image` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:29.781310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f93dd-6185-4a81-bffe-93ed1e1a5efd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:29.782662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f93dd-6185-4a81-bffe-93ed1e1a5efd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7b8f93dd-6185-4a81-bffe-93ed1e1a5efd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:29.791994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:29.792987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 07:25:29.793872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:29.801135 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 520/1048] 10.222.0.40 () {56 vars in 1105 bytes} [Mon Aug 1 07:25:29 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:29.801269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671470c-5235-4538-b484-2969d7a29013 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:29.882613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:29.905523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671470c-5235-4538-b484-2969d7a29013 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:29.941450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbaffa-a14c-4ad4-b183-6f2685e5117f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:29.942068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbaffa-a14c-4ad4-b183-6f2685e5117f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:29.942353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbaffa-a14c-4ad4-b183-6f2685e5117f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:29.942629 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbaffa-a14c-4ad4-b183-6f2685e5117f None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:29.943359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbaffa-a14c-4ad4-b183-6f2685e5117f None admin] RBAC: Adding query filter params (name=image) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:29.943660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbaffa-a14c-4ad4-b183-6f2685e5117f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:29.945050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbaffa-a14c-4ad4-b183-6f2685e5117f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:29.959511 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 529/1049] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:29.962470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f93dd-6185-4a81-bffe-93ed1e1a5efd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:30.063344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f93dd-6185-4a81-bffe-93ed1e1a5efd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:30.079374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:30.098684 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad0dba-4902-41dd-b3dc-e13efda2e5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:30.099337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad0dba-4902-41dd-b3dc-e13efda2e5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:30.100199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad0dba-4902-41dd-b3dc-e13efda2e5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:30.100473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad0dba-4902-41dd-b3dc-e13efda2e5e0 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:30.101236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad0dba-4902-41dd-b3dc-e13efda2e5e0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:30.101540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad0dba-4902-41dd-b3dc-e13efda2e5e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:30.102910 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad0dba-4902-41dd-b3dc-e13efda2e5e0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:30.108078 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 521/1050] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:30.113480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbaffa-a14c-4ad4-b183-6f2685e5117f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:30.213889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbaffa-a14c-4ad4-b183-6f2685e5117f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:30.246901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b9669-3694-40a2-930e-d79d90a4e2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:30.247541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b9669-3694-40a2-930e-d79d90a4e2b2 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:30.247822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b9669-3694-40a2-930e-d79d90a4e2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:30.248095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b9669-3694-40a2-930e-d79d90a4e2b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:30.248803 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b9669-3694-40a2-930e-d79d90a4e2b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:30.250192 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b9669-3694-40a2-930e-d79d90a4e2b2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:30.265665 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:30.279289 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 530/1051] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:25:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:25:30.588261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61750c0-a972-4eb2-9633-0214c86d44ae None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:30.588608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61750c0-a972-4eb2-9633-0214c86d44ae None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:30.588787 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61750c0-a972-4eb2-9633-0214c86d44ae None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:30.590120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 522/1052] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:30.595518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f5b02-2e34-4785-a465-a8eeb76f55bc None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:30.595817 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f5b02-2e34-4785-a465-a8eeb76f55bc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:30.596102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f5b02-2e34-4785-a465-a8eeb76f55bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:30.616402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-556f5b02-2e34-4785-a465-a8eeb76f55bc None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:30.617615 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-556f5b02-2e34-4785-a465-a8eeb76f55bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:30.696478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 531/1053] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:30.719912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61750c0-a972-4eb2-9633-0214c86d44ae None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:30.826454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61750c0-a972-4eb2-9633-0214c86d44ae None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:30.861391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5041263-75a3-460a-b70a-56bbf7db6937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:30.861950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5041263-75a3-460a-b70a-56bbf7db6937 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:30.862235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5041263-75a3-460a-b70a-56bbf7db6937 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:30.862518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5041263-75a3-460a-b70a-56bbf7db6937 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:30.874104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5041263-75a3-460a-b70a-56bbf7db6937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f5041263-75a3-460a-b70a-56bbf7db6937 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:30.879151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:30.880053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:30.881077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:30.881077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:30.881077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:30.881077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:30.881077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:30.881077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:30.881077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:30.881077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:30.881077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:30.881077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:30.881077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:25:30.881077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:30.885815 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 523/1054] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:25:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:30.887669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f5b02-2e34-4785-a465-a8eeb76f55bc None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:31.002578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f5b02-2e34-4785-a465-a8eeb76f55bc None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:31.028529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:31.041245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5a4c2-0e50-4702-b8b7-073930d58a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:31.041878 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5a4c2-0e50-4702-b8b7-073930d58a16 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:31.042163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5a4c2-0e50-4702-b8b7-073930d58a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:31.042442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5a4c2-0e50-4702-b8b7-073930d58a16 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:31.043187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5a4c2-0e50-4702-b8b7-073930d58a16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:31.043502 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5a4c2-0e50-4702-b8b7-073930d58a16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:31.044870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5a4c2-0e50-4702-b8b7-073930d58a16 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:31.054513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 532/1055] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:25:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:31.060194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5041263-75a3-460a-b70a-56bbf7db6937 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:31.108547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:31.171160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5041263-75a3-460a-b70a-56bbf7db6937 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:31.205409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15ef12-569c-436c-a389-00f3e921e560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:31.206046 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15ef12-569c-436c-a389-00f3e921e560 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:31.206332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15ef12-569c-436c-a389-00f3e921e560 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:31.206612 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15ef12-569c-436c-a389-00f3e921e560 None admin] PATH_INFO: `/v3/users/72ea3311fd8c45ebb07c9ec2e3f99836` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:31.220724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15ef12-569c-436c-a389-00f3e921e560 None admin] RBAC: Authorizing `identity:get_user(user_id=72ea3311fd8c45ebb07c9ec2e3f99836)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:31.222214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15ef12-569c-436c-a389-00f3e921e560 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:31.229564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 524/1056] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:25:31 2022] GET /identity/v3/users/72ea3311fd8c45ebb07c9ec2e3f99836 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:31.230967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5a4c2-0e50-4702-b8b7-073930d58a16 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:31.333104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5a4c2-0e50-4702-b8b7-073930d58a16 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:31.367148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562fa9f-f8e4-4bb0-b771-a378f50daa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:31.367770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562fa9f-f8e4-4bb0-b771-a378f50daa9b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:31.368047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562fa9f-f8e4-4bb0-b771-a378f50daa9b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:31.368320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562fa9f-f8e4-4bb0-b771-a378f50daa9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:31.422133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f261b62-fdb6-41aa-bbce-9d1689aab06a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:31.422405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f261b62-fdb6-41aa-bbce-9d1689aab06a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:31.422621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f261b62-fdb6-41aa-bbce-9d1689aab06a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:31.424137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 525/1057] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:25:31.425721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:31.429246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185ffd6-ab64-48c3-91a6-9688239bee4f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:31.429540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185ffd6-ab64-48c3-91a6-9688239bee4f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:31.429819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185ffd6-ab64-48c3-91a6-9688239bee4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:31.441219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562fa9f-f8e4-4bb0-b771-a378f50daa9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:31.443647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562fa9f-f8e4-4bb0-b771-a378f50daa9b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:31.447903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5185ffd6-ab64-48c3-91a6-9688239bee4f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:31.449106 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5185ffd6-ab64-48c3-91a6-9688239bee4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:31.522297 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 526/1058] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:31.544444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185ffd6-ab64-48c3-91a6-9688239bee4f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:31.563215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 533/1059] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:25:31.564252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562fa9f-f8e4-4bb0-b771-a378f50daa9b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:31.572916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:31.648194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185ffd6-ab64-48c3-91a6-9688239bee4f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:31.660950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:31.677185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562fa9f-f8e4-4bb0-b771-a378f50daa9b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:31.688157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ed174-4605-4d96-846a-6479cddc308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:31.688801 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ed174-4605-4d96-846a-6479cddc308b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:31.689085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ed174-4605-4d96-846a-6479cddc308b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:31.689368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ed174-4605-4d96-846a-6479cddc308b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:31.700531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ed174-4605-4d96-846a-6479cddc308b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-741ed174-4605-4d96-846a-6479cddc308b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:31.705402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:31.706316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:31.707168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:31.707168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:31.707168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:31.707168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:31.707168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:31.707168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:31.707168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:31.707168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:31.707168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:31.707168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:31.707168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:25:31.707168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:31.712140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 527/1060] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Aug 1 07:25:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:31.712884 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ed174-4605-4d96-846a-6479cddc308b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:31.716357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd374a-4d02-40b6-ac98-88dd5cb6093e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:31.717576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd374a-4d02-40b6-ac98-88dd5cb6093e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:31.717576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd374a-4d02-40b6-ac98-88dd5cb6093e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:31.717576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd374a-4d02-40b6-ac98-88dd5cb6093e None admin] PATH_INFO: `/v3/projects/6b06f96780cf48ffa922d449b321afc0` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:31.718710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd374a-4d02-40b6-ac98-88dd5cb6093e None admin] RBAC: Authorizing `identity:get_project(project_id=6b06f96780cf48ffa922d449b321afc0)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:31.720175 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd374a-4d02-40b6-ac98-88dd5cb6093e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:31.727527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 534/1061] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:25:31 2022] GET /identity/v3/projects/6b06f96780cf48ffa922d449b321afc0 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:31.728354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd374a-4d02-40b6-ac98-88dd5cb6093e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:31.811733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:31.824810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ed174-4605-4d96-846a-6479cddc308b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:31.836673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd374a-4d02-40b6-ac98-88dd5cb6093e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:31.860166 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c4b3a-774e-4294-af18-71dd0da3f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:31.860798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c4b3a-774e-4294-af18-71dd0da3f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:31.861066 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c4b3a-774e-4294-af18-71dd0da3f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:31.861338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c4b3a-774e-4294-af18-71dd0da3f2c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:31.862054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c4b3a-774e-4294-af18-71dd0da3f2c1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:31.862357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c4b3a-774e-4294-af18-71dd0da3f2c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:31.863748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c4b3a-774e-4294-af18-71dd0da3f2c1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:31.872155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 528/1062] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Aug 1 07:25:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:31.873713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0244ae-0c91-4a31-b0c9-98b2b53ff321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:31.874336 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0244ae-0c91-4a31-b0c9-98b2b53ff321 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:31.874622 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0244ae-0c91-4a31-b0c9-98b2b53ff321 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:31.874902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0244ae-0c91-4a31-b0c9-98b2b53ff321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:31.875728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0244ae-0c91-4a31-b0c9-98b2b53ff321 None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=6b06f96780cf48ffa922d449b321afc0, user.id=72ea3311fd8c45ebb07c9ec2e3f99836) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:31.876048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0244ae-0c91-4a31-b0c9-98b2b53ff321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:31.877594 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c4b3a-774e-4294-af18-71dd0da3f2c1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:31.877789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0244ae-0c91-4a31-b0c9-98b2b53ff321 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:31.885371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 535/1063] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:25:31 2022] GET /identity/v3/role_assignments?user.id=72ea3311fd8c45ebb07c9ec2e3f99836&scope.project.id=6b06f96780cf48ffa922d449b321afc0&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:32.006621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c4b3a-774e-4294-af18-71dd0da3f2c1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:32.045834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79215ae7-1208-4ab4-bb39-2f4e134c445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:32.046041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79215ae7-1208-4ab4-bb39-2f4e134c445b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:32.046265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79215ae7-1208-4ab4-bb39-2f4e134c445b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:32.046482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79215ae7-1208-4ab4-bb39-2f4e134c445b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:32.058733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79215ae7-1208-4ab4-bb39-2f4e134c445b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:32.060199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79215ae7-1208-4ab4-bb39-2f4e134c445b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-79215ae7-1208-4ab4-bb39-2f4e134c445b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:32.075474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:32.076444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:32.077350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:32.077350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:32.077350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:32.077350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:32.077350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 529/1064] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:32.082305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0244ae-0c91-4a31-b0c9-98b2b53ff321 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:32.103214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:32.186497 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0244ae-0c91-4a31-b0c9-98b2b53ff321 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:32.221090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e39eb4-1823-40f1-be5f-d5481809b078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:32.221639 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e39eb4-1823-40f1-be5f-d5481809b078 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:32.221924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e39eb4-1823-40f1-be5f-d5481809b078 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:32.222205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e39eb4-1823-40f1-be5f-d5481809b078 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:32.222896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e39eb4-1823-40f1-be5f-d5481809b078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:32.223239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e39eb4-1823-40f1-be5f-d5481809b078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:32.224636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e39eb4-1823-40f1-be5f-d5481809b078 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:32.241136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 536/1065] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:32.242434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79215ae7-1208-4ab4-bb39-2f4e134c445b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:32.245568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:32.314468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ae43e-795f-4d84-8bd3-1b5d18e97a32 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:32.314820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ae43e-795f-4d84-8bd3-1b5d18e97a32 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:32.315167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ae43e-795f-4d84-8bd3-1b5d18e97a32 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:32.316568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 537/1066] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:25:32.322470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4326eb4-e017-4762-91af-0ec922af3769 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:32.322759 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4326eb4-e017-4762-91af-0ec922af3769 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:32.323042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4326eb4-e017-4762-91af-0ec922af3769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:32.342546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b4326eb4-e017-4762-91af-0ec922af3769 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:32.343764 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b4326eb4-e017-4762-91af-0ec922af3769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:32.352557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79215ae7-1208-4ab4-bb39-2f4e134c445b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:32.388825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc1b5a-e6ba-446d-b962-971b933c9d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:32.389465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc1b5a-e6ba-446d-b962-971b933c9d70 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:32.389744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc1b5a-e6ba-446d-b962-971b933c9d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:32.390017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc1b5a-e6ba-446d-b962-971b933c9d70 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:32.397969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc1b5a-e6ba-446d-b962-971b933c9d70 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:32.399432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc1b5a-e6ba-446d-b962-971b933c9d70 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-23cc1b5a-e6ba-446d-b962-971b933c9d70 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:25:32.410757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:32.411759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:32.412810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:32.412810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:32.412810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:25:32.412810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:25:32.412810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:25:32.412810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:25:32.412810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:25:32.412810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:25:32.412810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:25:32.412810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:32.412810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 530/1067] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Aug 1 07:25:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:32.417727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc1b5a-e6ba-446d-b962-971b933c9d70 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:32.423106 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 538/1068] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:32.442586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4326eb4-e017-4762-91af-0ec922af3769 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:32.454508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:32.485019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:32.527462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc1b5a-e6ba-446d-b962-971b933c9d70 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:32.553628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4326eb4-e017-4762-91af-0ec922af3769 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:32.561695 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c940d8-a540-4a7f-9560-12409ae35e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:32.562311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c940d8-a540-4a7f-9560-12409ae35e82 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:32.562594 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c940d8-a540-4a7f-9560-12409ae35e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:32.562874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c940d8-a540-4a7f-9560-12409ae35e82 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:32.563734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c940d8-a540-4a7f-9560-12409ae35e82 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:32.564044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c940d8-a540-4a7f-9560-12409ae35e82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:32.565468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c940d8-a540-4a7f-9560-12409ae35e82 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:32.584056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 531/1069] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Aug 1 07:25:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:32.590261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c940d8-a540-4a7f-9560-12409ae35e82 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:32.595287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc0043-a8fd-4fa5-becb-de9238eac8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:32.595918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc0043-a8fd-4fa5-becb-de9238eac8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:32.596201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc0043-a8fd-4fa5-becb-de9238eac8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:32.596478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc0043-a8fd-4fa5-becb-de9238eac8d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:32.670553 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc0043-a8fd-4fa5-becb-de9238eac8d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:32.672039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc0043-a8fd-4fa5-becb-de9238eac8d5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:32.682020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:32.701580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c940d8-a540-4a7f-9560-12409ae35e82 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:32.721870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:32.737524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91294d0f-88c9-425d-b322-8ce324e0c66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:32.738101 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91294d0f-88c9-425d-b322-8ce324e0c66f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:32.738388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91294d0f-88c9-425d-b322-8ce324e0c66f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:32.738666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91294d0f-88c9-425d-b322-8ce324e0c66f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:32.749425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91294d0f-88c9-425d-b322-8ce324e0c66f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:32.750862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91294d0f-88c9-425d-b322-8ce324e0c66f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-91294d0f-88c9-425d-b322-8ce324e0c66f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:32.768799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:32.769670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:32.770552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:32.770552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:32.770552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:32.770552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:32.772437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 532/1070] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:32.775114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 539/1071] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 07:25:32.780377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91294d0f-88c9-425d-b322-8ce324e0c66f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:32.781860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc0043-a8fd-4fa5-becb-de9238eac8d5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:32.889788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc0043-a8fd-4fa5-becb-de9238eac8d5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:32.890387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91294d0f-88c9-425d-b322-8ce324e0c66f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:32.901957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:32.925820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eba05c-3dc5-4e24-9175-b8286f504c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:32.926450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eba05c-3dc5-4e24-9175-b8286f504c0e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:32.926720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eba05c-3dc5-4e24-9175-b8286f504c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:32.926997 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eba05c-3dc5-4e24-9175-b8286f504c0e None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:32.928280 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eba05c-3dc5-4e24-9175-b8286f504c0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:32.928589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eba05c-3dc5-4e24-9175-b8286f504c0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:32.929993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eba05c-3dc5-4e24-9175-b8286f504c0e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:32.931317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11b0ef-d5dd-442d-a764-3db60da8ebd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:32.931924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11b0ef-d5dd-442d-a764-3db60da8ebd1 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:32.932207 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11b0ef-d5dd-442d-a764-3db60da8ebd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:32.932775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11b0ef-d5dd-442d-a764-3db60da8ebd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:32.942789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11b0ef-d5dd-442d-a764-3db60da8ebd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:32.947873 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 533/1072] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:32.948017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11b0ef-d5dd-442d-a764-3db60da8ebd1 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:32.949637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eba05c-3dc5-4e24-9175-b8286f504c0e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-3c11b0ef-d5dd-442d-a764-3db60da8ebd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:32.963723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:32.964660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:32.966047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:32.966047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:32.966047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:32.966047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:32.969949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 540/1073] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:32.970612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11b0ef-d5dd-442d-a764-3db60da8ebd1 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:32.988611 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:33.070583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eba05c-3dc5-4e24-9175-b8286f504c0e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:33.083713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11b0ef-d5dd-442d-a764-3db60da8ebd1 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:33.106706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09b30f-f135-4930-ba97-d1fe30d33232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:33.107469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09b30f-f135-4930-ba97-d1fe30d33232 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:33.107979 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09b30f-f135-4930-ba97-d1fe30d33232 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:33.108679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09b30f-f135-4930-ba97-d1fe30d33232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:33.121293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09b30f-f135-4930-ba97-d1fe30d33232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:33.122699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09b30f-f135-4930-ba97-d1fe30d33232 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:33.122874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f21cf-69d3-4262-8671-f31a11d6b5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:33.123489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f21cf-69d3-4262-8671-f31a11d6b5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:33.123779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f21cf-69d3-4262-8671-f31a11d6b5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:33.124059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f21cf-69d3-4262-8671-f31a11d6b5a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:33.124754 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f21cf-69d3-4262-8671-f31a11d6b5a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:33.125067 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f21cf-69d3-4262-8671-f31a11d6b5a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:33.126416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f21cf-69d3-4262-8671-f31a11d6b5a5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ab09b30f-f135-4930-ba97-d1fe30d33232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:25:33.135239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:33.136162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:33.137051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:25:33.137051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:25:33.137051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:25:33.137051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:25:33.137051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:25:33.137051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:25:33.137051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:33.137051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:33.137051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 534/1074] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:33.141307 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 541/1075] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:33.141946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09b30f-f135-4930-ba97-d1fe30d33232 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:33.243764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:33.253898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09b30f-f135-4930-ba97-d1fe30d33232 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:33.290258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e6024-4a48-48d4-83ce-9d56b324140d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:33.291293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e6024-4a48-48d4-83ce-9d56b324140d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:33.291293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e6024-4a48-48d4-83ce-9d56b324140d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:33.291495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e6024-4a48-48d4-83ce-9d56b324140d None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:33.292269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e6024-4a48-48d4-83ce-9d56b324140d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:33.292495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e6024-4a48-48d4-83ce-9d56b324140d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:33.293942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e6024-4a48-48d4-83ce-9d56b324140d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:33.307218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 535/1076] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:25:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:33.313236 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f21cf-69d3-4262-8671-f31a11d6b5a5 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:33.338696 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:33.421526 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f21cf-69d3-4262-8671-f31a11d6b5a5 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:33.457042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d62ac-3209-4702-b7fd-03d326839dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:33.457722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d62ac-3209-4702-b7fd-03d326839dab None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:33.458008 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d62ac-3209-4702-b7fd-03d326839dab None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:33.458289 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d62ac-3209-4702-b7fd-03d326839dab None admin] PATH_INFO: `/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/abdcd6d09fee45c696449641ea36eae3/roles/d1e0bfc9cbb14751b3daabc5e2de9813` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:33.460611 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d62ac-3209-4702-b7fd-03d326839dab None admin] RBAC: Authorizing `identity:create_grant(project_id=08ddf7d6f2424a85a741d017497b49c9, user_id=abdcd6d09fee45c696449641ea36eae3, role_id=d1e0bfc9cbb14751b3daabc5e2de9813)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:33.462055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d62ac-3209-4702-b7fd-03d326839dab None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:33.476368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 542/1077] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:25:33 2022] PUT /identity/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/abdcd6d09fee45c696449641ea36eae3/roles/d1e0bfc9cbb14751b3daabc5e2de9813 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:25:33.904501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4c100-b50a-4c35-8331-9727a24eb0a0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:33.905030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4c100-b50a-4c35-8331-9727a24eb0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:33.905030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4c100-b50a-4c35-8331-9727a24eb0a0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:33.906390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 536/1078] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:33.911573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffeb27-63e7-4f93-87b2-89d5a65f1e10 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:33.911896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffeb27-63e7-4f93-87b2-89d5a65f1e10 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:33.912268 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffeb27-63e7-4f93-87b2-89d5a65f1e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:33.930616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f9ffeb27-63e7-4f93-87b2-89d5a65f1e10 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:33.931838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f9ffeb27-63e7-4f93-87b2-89d5a65f1e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:34.004766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 543/1079] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:34.027156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4c100-b50a-4c35-8331-9727a24eb0a0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:34.128363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4c100-b50a-4c35-8331-9727a24eb0a0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:34.162990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1058a0f-c549-44c4-b6b4-0de2353c92cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:34.163572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1058a0f-c549-44c4-b6b4-0de2353c92cf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:34.163858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1058a0f-c549-44c4-b6b4-0de2353c92cf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:34.164158 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1058a0f-c549-44c4-b6b4-0de2353c92cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:34.174212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1058a0f-c549-44c4-b6b4-0de2353c92cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a1058a0f-c549-44c4-b6b4-0de2353c92cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:34.179654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:34.180817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:34.181679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 537/1080] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:25:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:34.186721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffeb27-63e7-4f93-87b2-89d5a65f1e10 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:34.198199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:34.287579 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffeb27-63e7-4f93-87b2-89d5a65f1e10 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:34.320402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a292f-bef9-4f3e-a564-151e90ff67c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:34.321009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a292f-bef9-4f3e-a564-151e90ff67c8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:34.321288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a292f-bef9-4f3e-a564-151e90ff67c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:34.321560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a292f-bef9-4f3e-a564-151e90ff67c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:34.322260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a292f-bef9-4f3e-a564-151e90ff67c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:34.322571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a292f-bef9-4f3e-a564-151e90ff67c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:34.323932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a292f-bef9-4f3e-a564-151e90ff67c8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:34.331768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 544/1081] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:25:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:34.337833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1058a0f-c549-44c4-b6b4-0de2353c92cf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:34.353115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:34.436811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1058a0f-c549-44c4-b6b4-0de2353c92cf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:34.469452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40543034-7dc9-48be-9292-912517826ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:34.470072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40543034-7dc9-48be-9292-912517826ebf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:34.470353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40543034-7dc9-48be-9292-912517826ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:34.470630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40543034-7dc9-48be-9292-912517826ebf None admin] PATH_INFO: `/v3/users/72ea3311fd8c45ebb07c9ec2e3f99836` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:34.471792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40543034-7dc9-48be-9292-912517826ebf None admin] RBAC: Authorizing `identity:get_user(user_id=72ea3311fd8c45ebb07c9ec2e3f99836)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:34.473207 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40543034-7dc9-48be-9292-912517826ebf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:34.475629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 538/1082] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:25:34 2022] GET /identity/v3/users/72ea3311fd8c45ebb07c9ec2e3f99836 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:34.481213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a292f-bef9-4f3e-a564-151e90ff67c8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:34.550020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:34.582534 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a292f-bef9-4f3e-a564-151e90ff67c8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:34.615944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092924f8-a0e8-47af-baeb-f730142aa982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:34.616582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092924f8-a0e8-47af-baeb-f730142aa982 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:34.616862 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092924f8-a0e8-47af-baeb-f730142aa982 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:34.617139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092924f8-a0e8-47af-baeb-f730142aa982 None admin] PATH_INFO: `/v3/projects/6b06f96780cf48ffa922d449b321afc0` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:34.618276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092924f8-a0e8-47af-baeb-f730142aa982 None admin] RBAC: Authorizing `identity:get_project(project_id=6b06f96780cf48ffa922d449b321afc0)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:34.619735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092924f8-a0e8-47af-baeb-f730142aa982 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:34.621387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 545/1083] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:25:34 2022] GET /identity/v3/projects/6b06f96780cf48ffa922d449b321afc0 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:34.627344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40543034-7dc9-48be-9292-912517826ebf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:34.707225 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:34.727205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40543034-7dc9-48be-9292-912517826ebf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:34.759637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12351f9-c672-4769-acbd-f829de89a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:34.760291 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12351f9-c672-4769-acbd-f829de89a9fc None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:34.760573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12351f9-c672-4769-acbd-f829de89a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:34.760949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12351f9-c672-4769-acbd-f829de89a9fc None admin] PATH_INFO: `/v3/projects/6b06f96780cf48ffa922d449b321afc0/users/72ea3311fd8c45ebb07c9ec2e3f99836/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:34.762828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12351f9-c672-4769-acbd-f829de89a9fc None admin] RBAC: Authorizing `identity:create_grant(project_id=6b06f96780cf48ffa922d449b321afc0, user_id=72ea3311fd8c45ebb07c9ec2e3f99836, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:34.764289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12351f9-c672-4769-acbd-f829de89a9fc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:34.777155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 539/1084] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:25:34 2022] PUT /identity/v3/projects/6b06f96780cf48ffa922d449b321afc0/users/72ea3311fd8c45ebb07c9ec2e3f99836/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:25:35.468804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5b44a-4633-4545-9d8c-54f33b71a368 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:35.469351 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5b44a-4633-4545-9d8c-54f33b71a368 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:35.469447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5b44a-4633-4545-9d8c-54f33b71a368 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:35.470825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 546/1085] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:35.476258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f71a88-1dcd-40d0-bb32-21cb3a31ecee None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:35.476648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f71a88-1dcd-40d0-bb32-21cb3a31ecee None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:35.476930 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f71a88-1dcd-40d0-bb32-21cb3a31ecee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:35.495205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f3f71a88-1dcd-40d0-bb32-21cb3a31ecee None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:35.496387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f3f71a88-1dcd-40d0-bb32-21cb3a31ecee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:35.560364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 540/1086] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:35.583493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5b44a-4633-4545-9d8c-54f33b71a368 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:35.683319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5b44a-4633-4545-9d8c-54f33b71a368 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:35.715638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a5f5a-39ec-47a6-9a2e-0e5005bb7d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:35.716271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a5f5a-39ec-47a6-9a2e-0e5005bb7d72 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:35.716557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a5f5a-39ec-47a6-9a2e-0e5005bb7d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:35.716830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a5f5a-39ec-47a6-9a2e-0e5005bb7d72 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:35.724747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:35.728229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a5f5a-39ec-47a6-9a2e-0e5005bb7d72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-872a5f5a-39ec-47a6-9a2e-0e5005bb7d72 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:35.733411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:35.734532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:35.735437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 547/1087] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Aug 1 07:25:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:35.741179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f71a88-1dcd-40d0-bb32-21cb3a31ecee None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:35.810670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:35.841834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f71a88-1dcd-40d0-bb32-21cb3a31ecee None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:35.877477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c178895-76fa-430a-928d-70649b4eb613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:35.878100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c178895-76fa-430a-928d-70649b4eb613 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:35.878383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c178895-76fa-430a-928d-70649b4eb613 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:35.878657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c178895-76fa-430a-928d-70649b4eb613 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:35.879403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c178895-76fa-430a-928d-70649b4eb613 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:35.879710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c178895-76fa-430a-928d-70649b4eb613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:35.881109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c178895-76fa-430a-928d-70649b4eb613 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:35.889351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 541/1088] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Aug 1 07:25:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:35.894972 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a5f5a-39ec-47a6-9a2e-0e5005bb7d72 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:35.998816 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a5f5a-39ec-47a6-9a2e-0e5005bb7d72 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:36.033417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b0806-e16a-47ab-855e-d0afd0eff687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:36.034047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b0806-e16a-47ab-855e-d0afd0eff687 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:36.034334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b0806-e16a-47ab-855e-d0afd0eff687 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:36.034615 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b0806-e16a-47ab-855e-d0afd0eff687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:36.045533 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b0806-e16a-47ab-855e-d0afd0eff687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:36.046966 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b0806-e16a-47ab-855e-d0afd0eff687 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-505b0806-e16a-47ab-855e-d0afd0eff687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:36.061369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:36.062434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:36.063408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:36.063408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:36.063408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:36.063408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:36.063408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 548/1089] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:36.068777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c178895-76fa-430a-928d-70649b4eb613 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:36.168028 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c178895-76fa-430a-928d-70649b4eb613 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:36.196352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:36.201606 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac40bcd-a856-4db2-8e7a-d17800aa3d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:36.202222 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac40bcd-a856-4db2-8e7a-d17800aa3d21 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:36.202511 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac40bcd-a856-4db2-8e7a-d17800aa3d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:36.202792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac40bcd-a856-4db2-8e7a-d17800aa3d21 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:36.203514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac40bcd-a856-4db2-8e7a-d17800aa3d21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:36.203825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac40bcd-a856-4db2-8e7a-d17800aa3d21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:36.205188 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac40bcd-a856-4db2-8e7a-d17800aa3d21 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:36.217456 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 542/1090] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:36.223115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b0806-e16a-47ab-855e-d0afd0eff687 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:36.275620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:36.322525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b0806-e16a-47ab-855e-d0afd0eff687 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:36.356728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d0ef7-3ef2-4df3-b377-aaf5dcae5cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:36.357377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d0ef7-3ef2-4df3-b377-aaf5dcae5cce None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:36.357663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d0ef7-3ef2-4df3-b377-aaf5dcae5cce None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:36.357942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d0ef7-3ef2-4df3-b377-aaf5dcae5cce None admin] PATH_INFO: `/v3/users/nova` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:36.365304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d0ef7-3ef2-4df3-b377-aaf5dcae5cce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:36.366738 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d0ef7-3ef2-4df3-b377-aaf5dcae5cce None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-e50d0ef7-3ef2-4df3-b377-aaf5dcae5cce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:25:36.377775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:36.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:36.380029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:36.380029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:36.380029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:25:36.380029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:25:36.380029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:25:36.380029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:25:36.380029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:25:36.380029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:25:36.380029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 07:25:36.380029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:36.380029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 549/1091] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Aug 1 07:25:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:36.384888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac40bcd-a856-4db2-8e7a-d17800aa3d21 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:36.482390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac40bcd-a856-4db2-8e7a-d17800aa3d21 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:36.515835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0a3e1-fbed-44cd-97bc-3df8c3c581ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:36.516459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0a3e1-fbed-44cd-97bc-3df8c3c581ec None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:36.516745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0a3e1-fbed-44cd-97bc-3df8c3c581ec None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:36.517022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0a3e1-fbed-44cd-97bc-3df8c3c581ec None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:36.517813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0a3e1-fbed-44cd-97bc-3df8c3c581ec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:36.518124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0a3e1-fbed-44cd-97bc-3df8c3c581ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:36.519585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0a3e1-fbed-44cd-97bc-3df8c3c581ec None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:36.547622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 543/1092] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Aug 1 07:25:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:36.547792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d0ef7-3ef2-4df3-b377-aaf5dcae5cce None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:36.647519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d0ef7-3ef2-4df3-b377-aaf5dcae5cce None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:36.659913 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:36.682470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e6448-f1d2-4ebf-8f1e-86f95352bb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:36.683122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e6448-f1d2-4ebf-8f1e-86f95352bb18 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:36.683411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e6448-f1d2-4ebf-8f1e-86f95352bb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:36.683690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e6448-f1d2-4ebf-8f1e-86f95352bb18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:36.749276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e6448-f1d2-4ebf-8f1e-86f95352bb18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:36.750734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e6448-f1d2-4ebf-8f1e-86f95352bb18 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:36.758572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec820001-d80e-4d76-9bf9-7f894b760427 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:36.758873 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec820001-d80e-4d76-9bf9-7f894b760427 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:36.759189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec820001-d80e-4d76-9bf9-7f894b760427 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:36.760544 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 544/1093] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:25:36.765407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebba0a-e547-49af-8a0a-3bdea5b8940a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:36.765699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebba0a-e547-49af-8a0a-3bdea5b8940a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:36.765972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebba0a-e547-49af-8a0a-3bdea5b8940a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:36.784903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-58ebba0a-e547-49af-8a0a-3bdea5b8940a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:36.786200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-58ebba0a-e547-49af-8a0a-3bdea5b8940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:36.832390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:36.850514 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 550/1094] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:25:36.855582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 545/1095] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:36.856284 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e6448-f1d2-4ebf-8f1e-86f95352bb18 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:36.870488 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:36.876760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebba0a-e547-49af-8a0a-3bdea5b8940a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:36.955156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e6448-f1d2-4ebf-8f1e-86f95352bb18 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:36.979155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebba0a-e547-49af-8a0a-3bdea5b8940a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:36.991031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3659a7-4a19-4fa3-aa8d-9588af020187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:36.991704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3659a7-4a19-4fa3-aa8d-9588af020187 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:36.991988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3659a7-4a19-4fa3-aa8d-9588af020187 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:36.992263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3659a7-4a19-4fa3-aa8d-9588af020187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:37.002097 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3659a7-4a19-4fa3-aa8d-9588af020187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:37.003559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3659a7-4a19-4fa3-aa8d-9588af020187 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:37.014246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bcea1-bc08-4bc1-a4ec-0743d99c54a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:37.014883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bcea1-bc08-4bc1-a4ec-0743d99c54a5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:37.015206 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bcea1-bc08-4bc1-a4ec-0743d99c54a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:37.015487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bcea1-bc08-4bc1-a4ec-0743d99c54a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-2c3659a7-4a19-4fa3-aa8d-9588af020187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:25:37.019034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:37.019931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:37.020844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:25:37.020844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:25:37.020844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:25:37.020844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:37.020844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 551/1096] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:37.026517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3659a7-4a19-4fa3-aa8d-9588af020187 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:37.026628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175bcea1-bc08-4bc1-a4ec-0743d99c54a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-175bcea1-bc08-4bc1-a4ec-0743d99c54a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:37.031404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:37.032384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:37.033215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 546/1097] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:25:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:37.038489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bcea1-bc08-4bc1-a4ec-0743d99c54a5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:37.081807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:37.091483 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:37.135818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3659a7-4a19-4fa3-aa8d-9588af020187 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:37.149192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bcea1-bc08-4bc1-a4ec-0743d99c54a5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:37.176662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a241ae5-b035-4f4b-aab1-998439db7b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:37.177955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a241ae5-b035-4f4b-aab1-998439db7b53 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:37.178358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a241ae5-b035-4f4b-aab1-998439db7b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:37.178647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a241ae5-b035-4f4b-aab1-998439db7b53 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:37.179363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a241ae5-b035-4f4b-aab1-998439db7b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:37.179676 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a241ae5-b035-4f4b-aab1-998439db7b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:37.181045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a241ae5-b035-4f4b-aab1-998439db7b53 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:37.183615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d782e-031c-4777-b9fd-9added6d28cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:37.184226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d782e-031c-4777-b9fd-9added6d28cb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:37.184509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d782e-031c-4777-b9fd-9added6d28cb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:37.184780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d782e-031c-4777-b9fd-9added6d28cb None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:37.185486 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d782e-031c-4777-b9fd-9added6d28cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:37.185791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d782e-031c-4777-b9fd-9added6d28cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:37.187152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d782e-031c-4777-b9fd-9added6d28cb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:37.192844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 552/1098] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:37.195265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 547/1099] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:25:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:37.198411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a241ae5-b035-4f4b-aab1-998439db7b53 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:37.200937 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d782e-031c-4777-b9fd-9added6d28cb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:37.295866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a241ae5-b035-4f4b-aab1-998439db7b53 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:37.300221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d782e-031c-4777-b9fd-9added6d28cb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:37.301742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:37.319872 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:37.330679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc55e9-d6d0-426d-b05e-c0af8c635af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:37.331338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc55e9-d6d0-426d-b05e-c0af8c635af4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:37.331619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc55e9-d6d0-426d-b05e-c0af8c635af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:37.331896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc55e9-d6d0-426d-b05e-c0af8c635af4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:37.336391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e021e2d-eb72-4f0e-849f-f67c6a3d34e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:37.337024 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e021e2d-eb72-4f0e-849f-f67c6a3d34e7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:37.337310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e021e2d-eb72-4f0e-849f-f67c6a3d34e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:37.337589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e021e2d-eb72-4f0e-849f-f67c6a3d34e7 None admin] PATH_INFO: `/v3/users/72ea3311fd8c45ebb07c9ec2e3f99836` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:37.338811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e021e2d-eb72-4f0e-849f-f67c6a3d34e7 None admin] RBAC: Authorizing `identity:get_user(user_id=72ea3311fd8c45ebb07c9ec2e3f99836)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:37.338928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc55e9-d6d0-426d-b05e-c0af8c635af4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:37.340286 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc55e9-d6d0-426d-b05e-c0af8c635af4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:37.340380 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e021e2d-eb72-4f0e-849f-f67c6a3d34e7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:37.341978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 548/1100] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:25:37 2022] GET /identity/v3/users/72ea3311fd8c45ebb07c9ec2e3f99836 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:37.347522 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e021e2d-eb72-4f0e-849f-f67c6a3d34e7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-57bc55e9-d6d0-426d-b05e-c0af8c635af4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:25:37.352891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:37.353805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:37.354792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:25:37.354792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:25:37.354792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:25:37.354792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:25:37.354792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:25:37.354792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:25:37.354792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:25:37.354792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:25:37.354792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 553/1101] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:25:37.359654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc55e9-d6d0-426d-b05e-c0af8c635af4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:37.447329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e021e2d-eb72-4f0e-849f-f67c6a3d34e7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:37.462658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc55e9-d6d0-426d-b05e-c0af8c635af4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:37.480970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaeb9dd-dd74-4c05-8166-2aa27c3e088f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:37.481600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeb9dd-dd74-4c05-8166-2aa27c3e088f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:37.481891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeb9dd-dd74-4c05-8166-2aa27c3e088f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:37.482166 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeb9dd-dd74-4c05-8166-2aa27c3e088f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:37.496480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0f3c5-da2e-4e58-9dbc-568bdfea874a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:37.497086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0f3c5-da2e-4e58-9dbc-568bdfea874a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:37.497368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0f3c5-da2e-4e58-9dbc-568bdfea874a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:37.497643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0f3c5-da2e-4e58-9dbc-568bdfea874a None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:37.498377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0f3c5-da2e-4e58-9dbc-568bdfea874a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:37.498692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0f3c5-da2e-4e58-9dbc-568bdfea874a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:37.500158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0f3c5-da2e-4e58-9dbc-568bdfea874a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:37.513214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 554/1102] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:37.519158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0f3c5-da2e-4e58-9dbc-568bdfea874a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:37.521621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:37.550031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaeb9dd-dd74-4c05-8166-2aa27c3e088f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:37.551521 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaeb9dd-dd74-4c05-8166-2aa27c3e088f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:37.563778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:37.618955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0f3c5-da2e-4e58-9dbc-568bdfea874a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:37.651917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 549/1103] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 07:25:37.657056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86095925-52e7-4efe-8728-e1a96d73f177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:37.657056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86095925-52e7-4efe-8728-e1a96d73f177 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:37.657056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86095925-52e7-4efe-8728-e1a96d73f177 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:37.657056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86095925-52e7-4efe-8728-e1a96d73f177 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:37.657410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86095925-52e7-4efe-8728-e1a96d73f177 None admin] RBAC: Adding query filter params (role.id=d1e0bfc9cbb14751b3daabc5e2de9813, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=abdcd6d09fee45c696449641ea36eae3) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:37.657410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86095925-52e7-4efe-8728-e1a96d73f177 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:37.657711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86095925-52e7-4efe-8728-e1a96d73f177 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:37.657813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaeb9dd-dd74-4c05-8166-2aa27c3e088f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:37.664541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 555/1104] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:25:37 2022] GET /identity/v3/role_assignments?user.id=abdcd6d09fee45c696449641ea36eae3&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=d1e0bfc9cbb14751b3daabc5e2de9813 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:37.731168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:37.764909 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaeb9dd-dd74-4c05-8166-2aa27c3e088f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:37.799672 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6601c7c-fb4e-4463-8ec4-aaac27b96ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:37.800327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6601c7c-fb4e-4463-8ec4-aaac27b96ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:37.800614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6601c7c-fb4e-4463-8ec4-aaac27b96ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:37.800895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6601c7c-fb4e-4463-8ec4-aaac27b96ac8 None admin] PATH_INFO: `/v3/projects/6b06f96780cf48ffa922d449b321afc0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:37.802626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6601c7c-fb4e-4463-8ec4-aaac27b96ac8 None admin] RBAC: Authorizing `identity:get_project(project_id=6b06f96780cf48ffa922d449b321afc0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:37.804111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6601c7c-fb4e-4463-8ec4-aaac27b96ac8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:37.805834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 550/1105] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:25:37 2022] GET /identity/v3/projects/6b06f96780cf48ffa922d449b321afc0 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:25:37.811991 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86095925-52e7-4efe-8728-e1a96d73f177 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:37.916353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86095925-52e7-4efe-8728-e1a96d73f177 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:37.925201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:37.953555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf240e0-48b3-4fcd-89d4-3a5a34ff2444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:37.953786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf240e0-48b3-4fcd-89d4-3a5a34ff2444 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:37.953884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf240e0-48b3-4fcd-89d4-3a5a34ff2444 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:37.954141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf240e0-48b3-4fcd-89d4-3a5a34ff2444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:37.954930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf240e0-48b3-4fcd-89d4-3a5a34ff2444 None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=6b06f96780cf48ffa922d449b321afc0, user.id=72ea3311fd8c45ebb07c9ec2e3f99836) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:37.955283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf240e0-48b3-4fcd-89d4-3a5a34ff2444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:37.956709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf240e0-48b3-4fcd-89d4-3a5a34ff2444 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:37.963825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 556/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:25:37 2022] GET /identity/v3/role_assignments?user.id=72ea3311fd8c45ebb07c9ec2e3f99836&scope.project.id=6b06f96780cf48ffa922d449b321afc0&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:40.016196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d0477-de45-4ace-b761-8691772bf8db None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:40.016722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d0477-de45-4ace-b761-8691772bf8db None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:40.016722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d0477-de45-4ace-b761-8691772bf8db None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:40.018094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 551/1107] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:40.023083 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c79614-71b4-4f9c-85a8-cb7bf5efb389 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:40.023494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c79614-71b4-4f9c-85a8-cb7bf5efb389 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:40.023653 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c79614-71b4-4f9c-85a8-cb7bf5efb389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:40.041427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-70c79614-71b4-4f9c-85a8-cb7bf5efb389 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:40.042601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-70c79614-71b4-4f9c-85a8-cb7bf5efb389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:40.108320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 557/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:25:40.132563 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d0477-de45-4ace-b761-8691772bf8db None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:40.230150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d0477-de45-4ace-b761-8691772bf8db None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:40.263250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1f406-00e7-461c-97b5-43c3505924b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:40.263860 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f406-00e7-461c-97b5-43c3505924b6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:40.264141 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f406-00e7-461c-97b5-43c3505924b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:40.264412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f406-00e7-461c-97b5-43c3505924b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:40.265533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1f406-00e7-461c-97b5-43c3505924b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:40.266951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1f406-00e7-461c-97b5-43c3505924b6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:40.268545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 552/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:40.274139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c79614-71b4-4f9c-85a8-cb7bf5efb389 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:40.371332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c79614-71b4-4f9c-85a8-cb7bf5efb389 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:40.372804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:40.405518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15191060-5dba-4706-8465-a27260dae3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:40.406123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15191060-5dba-4706-8465-a27260dae3db None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:40.406406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15191060-5dba-4706-8465-a27260dae3db None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:40.406683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15191060-5dba-4706-8465-a27260dae3db None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:40.407387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15191060-5dba-4706-8465-a27260dae3db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:40.408809 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15191060-5dba-4706-8465-a27260dae3db None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:40.430382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 558/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:25:40 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:25:42.495266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270eb1d-d40e-41d0-be90-23241f5452f2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:42.495895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270eb1d-d40e-41d0-be90-23241f5452f2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:42.495895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270eb1d-d40e-41d0-be90-23241f5452f2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:42.497162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 553/1111] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:42.502449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341b195-9cca-41dd-971e-bc8c11263fdc None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:42.502839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341b195-9cca-41dd-971e-bc8c11263fdc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:42.503006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341b195-9cca-41dd-971e-bc8c11263fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:42.522856 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c341b195-9cca-41dd-971e-bc8c11263fdc None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:42.524072 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c341b195-9cca-41dd-971e-bc8c11263fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:42.588056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 559/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:42.610530 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270eb1d-d40e-41d0-be90-23241f5452f2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:42.619271 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:42.707614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270eb1d-d40e-41d0-be90-23241f5452f2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:42.739902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1b6c3-264b-4b2a-bb6d-24bf2f92bb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:42.740527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1b6c3-264b-4b2a-bb6d-24bf2f92bb3a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:42.740808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1b6c3-264b-4b2a-bb6d-24bf2f92bb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:42.741084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1b6c3-264b-4b2a-bb6d-24bf2f92bb3a None admin] PATH_INFO: `/v3/roles/0a56be0718ea495f8a241ae2447069cf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:42.742500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1b6c3-264b-4b2a-bb6d-24bf2f92bb3a None admin] RBAC: Authorizing `identity:get_role(role_id=0a56be0718ea495f8a241ae2447069cf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:42.743880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1b6c3-264b-4b2a-bb6d-24bf2f92bb3a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:42.745206 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 554/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:25:42 2022] GET /identity/v3/roles/0a56be0718ea495f8a241ae2447069cf => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:42.751075 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341b195-9cca-41dd-971e-bc8c11263fdc None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:42.836852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:42.849548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341b195-9cca-41dd-971e-bc8c11263fdc None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:42.883627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702f21c-177f-4d06-ba36-fc2f18e98418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:42.884254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702f21c-177f-4d06-ba36-fc2f18e98418 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:42.884539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702f21c-177f-4d06-ba36-fc2f18e98418 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:42.884817 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702f21c-177f-4d06-ba36-fc2f18e98418 None admin] PATH_INFO: `/v3/users/6a095323aea74c4dad75ac737736c648` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:42.898033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702f21c-177f-4d06-ba36-fc2f18e98418 None admin] RBAC: Authorizing `identity:get_user(user_id=6a095323aea74c4dad75ac737736c648)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:42.899497 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702f21c-177f-4d06-ba36-fc2f18e98418 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:42.901116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 560/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:25:42 2022] GET /identity/v3/users/6a095323aea74c4dad75ac737736c648 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:42.906823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1b6c3-264b-4b2a-bb6d-24bf2f92bb3a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:42.980701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:43.007008 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1b6c3-264b-4b2a-bb6d-24bf2f92bb3a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:43.039948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb34e9-0ecd-4375-9a56-a1bc625b07f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:43.040545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb34e9-0ecd-4375-9a56-a1bc625b07f8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:43.040831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb34e9-0ecd-4375-9a56-a1bc625b07f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:43.041110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb34e9-0ecd-4375-9a56-a1bc625b07f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:43.106457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb34e9-0ecd-4375-9a56-a1bc625b07f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:43.107921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb34e9-0ecd-4375-9a56-a1bc625b07f8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:43.180674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:43.206242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 555/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:25:43.212306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702f21c-177f-4d06-ba36-fc2f18e98418 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:43.309212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702f21c-177f-4d06-ba36-fc2f18e98418 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:43.341544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2d129-87ec-4e04-8ef5-c78492639768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:43.342172 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2d129-87ec-4e04-8ef5-c78492639768 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:43.342459 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2d129-87ec-4e04-8ef5-c78492639768 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:43.342740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2d129-87ec-4e04-8ef5-c78492639768 None admin] PATH_INFO: `/v3/projects/6b06f96780cf48ffa922d449b321afc0` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:43.343905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2d129-87ec-4e04-8ef5-c78492639768 None admin] RBAC: Authorizing `identity:get_project(project_id=6b06f96780cf48ffa922d449b321afc0)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:43.345335 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2d129-87ec-4e04-8ef5-c78492639768 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:43.346976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 561/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:25:43 2022] GET /identity/v3/projects/6b06f96780cf48ffa922d449b321afc0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:43.353508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb34e9-0ecd-4375-9a56-a1bc625b07f8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:43.450102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb34e9-0ecd-4375-9a56-a1bc625b07f8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:43.483543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020e65c-da57-4a44-9eec-1d8a8edd529d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:43.484137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020e65c-da57-4a44-9eec-1d8a8edd529d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:43.484419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020e65c-da57-4a44-9eec-1d8a8edd529d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:43.484694 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020e65c-da57-4a44-9eec-1d8a8edd529d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:43.485475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020e65c-da57-4a44-9eec-1d8a8edd529d None admin] RBAC: Adding query filter params (role.id=0a56be0718ea495f8a241ae2447069cf, scope.project.id=6b06f96780cf48ffa922d449b321afc0, user.id=6a095323aea74c4dad75ac737736c648) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:43.485782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020e65c-da57-4a44-9eec-1d8a8edd529d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:43.487226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020e65c-da57-4a44-9eec-1d8a8edd529d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:43.492511 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 556/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:25:43 2022] GET /identity/v3/role_assignments?user.id=6a095323aea74c4dad75ac737736c648&scope.project.id=6b06f96780cf48ffa922d449b321afc0&role.id=0a56be0718ea495f8a241ae2447069cf => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:45.436556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863614b-ac14-432f-971d-13272c6ca6bd None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:45.437088 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863614b-ac14-432f-971d-13272c6ca6bd None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:45.437183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863614b-ac14-432f-971d-13272c6ca6bd None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:45.438467 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 562/1118] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:45.444427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ad8a7-ec3b-44c7-a810-dd2eae6dccdd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:45.444924 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ad8a7-ec3b-44c7-a810-dd2eae6dccdd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:45.445023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ad8a7-ec3b-44c7-a810-dd2eae6dccdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:45.463361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b88ad8a7-ec3b-44c7-a810-dd2eae6dccdd None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:45.464549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b88ad8a7-ec3b-44c7-a810-dd2eae6dccdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:45.492884 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:45.530084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 557/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:45.552342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863614b-ac14-432f-971d-13272c6ca6bd None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:45.568458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:45.652606 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863614b-ac14-432f-971d-13272c6ca6bd None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:45.684773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc49536-86ca-4911-832b-0204e2e6f1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:45.685402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc49536-86ca-4911-832b-0204e2e6f1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:45.685685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc49536-86ca-4911-832b-0204e2e6f1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:45.685966 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc49536-86ca-4911-832b-0204e2e6f1d0 None admin] PATH_INFO: `/v3/roles/0a56be0718ea495f8a241ae2447069cf` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:45.687459 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc49536-86ca-4911-832b-0204e2e6f1d0 None admin] RBAC: Authorizing `identity:get_role(role_id=0a56be0718ea495f8a241ae2447069cf)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:45.688781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc49536-86ca-4911-832b-0204e2e6f1d0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:45.690142 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 563/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:25:45 2022] GET /identity/v3/roles/0a56be0718ea495f8a241ae2447069cf => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:45.696381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ad8a7-ec3b-44c7-a810-dd2eae6dccdd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:45.792085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ad8a7-ec3b-44c7-a810-dd2eae6dccdd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:45.826182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc098cce-5b96-4fdf-b544-29b64cc54289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:45.826807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc098cce-5b96-4fdf-b544-29b64cc54289 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:45.827106 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc098cce-5b96-4fdf-b544-29b64cc54289 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:45.827388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc098cce-5b96-4fdf-b544-29b64cc54289 None admin] PATH_INFO: `/v3/users/6a095323aea74c4dad75ac737736c648` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:45.828521 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc098cce-5b96-4fdf-b544-29b64cc54289 None admin] RBAC: Authorizing `identity:get_user(user_id=6a095323aea74c4dad75ac737736c648)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:45.829938 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc098cce-5b96-4fdf-b544-29b64cc54289 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:45.831569 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 558/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:25:45 2022] GET /identity/v3/users/6a095323aea74c4dad75ac737736c648 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:45.837792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc49536-86ca-4911-832b-0204e2e6f1d0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:45.917688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:45.936083 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc49536-86ca-4911-832b-0204e2e6f1d0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:45.969418 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d8231-2929-4048-a256-e40fe8b79ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:45.970061 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d8231-2929-4048-a256-e40fe8b79ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:45.970346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d8231-2929-4048-a256-e40fe8b79ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:45.970623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d8231-2929-4048-a256-e40fe8b79ff3 None admin] PATH_INFO: `/v3/projects/6b06f96780cf48ffa922d449b321afc0` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:45.971785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d8231-2929-4048-a256-e40fe8b79ff3 None admin] RBAC: Authorizing `identity:get_project(project_id=6b06f96780cf48ffa922d449b321afc0)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:45.973230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d8231-2929-4048-a256-e40fe8b79ff3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:45.975119 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 564/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:25:45 2022] GET /identity/v3/projects/6b06f96780cf48ffa922d449b321afc0 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:45.980966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc098cce-5b96-4fdf-b544-29b64cc54289 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:46.015761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:46.078002 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc098cce-5b96-4fdf-b544-29b64cc54289 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:46.110672 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fee7e7-8ec2-4f0d-9959-ce451dd25575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:46.111348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fee7e7-8ec2-4f0d-9959-ce451dd25575 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:46.111632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fee7e7-8ec2-4f0d-9959-ce451dd25575 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:46.111909 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fee7e7-8ec2-4f0d-9959-ce451dd25575 None admin] PATH_INFO: `/v3/projects/6b06f96780cf48ffa922d449b321afc0/users/6a095323aea74c4dad75ac737736c648/roles/0a56be0718ea495f8a241ae2447069cf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:46.114554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fee7e7-8ec2-4f0d-9959-ce451dd25575 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b06f96780cf48ffa922d449b321afc0, user_id=6a095323aea74c4dad75ac737736c648, role_id=0a56be0718ea495f8a241ae2447069cf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:46.116092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fee7e7-8ec2-4f0d-9959-ce451dd25575 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:46.128725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 559/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:25:45 2022] PUT /identity/v3/projects/6b06f96780cf48ffa922d449b321afc0/users/6a095323aea74c4dad75ac737736c648/roles/0a56be0718ea495f8a241ae2447069cf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:25:48.085854 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a20617-d89f-4c60-9020-fedcc97d9482 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:48.086389 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a20617-d89f-4c60-9020-fedcc97d9482 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:48.086487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a20617-d89f-4c60-9020-fedcc97d9482 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:48.087823 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 565/1124] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:48.092979 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a79097-56f2-4f25-b0dd-9fff30d09d7b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:48.093412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a79097-56f2-4f25-b0dd-9fff30d09d7b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:48.093539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a79097-56f2-4f25-b0dd-9fff30d09d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:48.111070 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-71a79097-56f2-4f25-b0dd-9fff30d09d7b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:48.112270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-71a79097-56f2-4f25-b0dd-9fff30d09d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:48.175791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 560/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:48.199840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a20617-d89f-4c60-9020-fedcc97d9482 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:48.298803 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a20617-d89f-4c60-9020-fedcc97d9482 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:48.331756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a12a8-723d-467a-9dff-e80f1e94f811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:48.332373 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a12a8-723d-467a-9dff-e80f1e94f811 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:48.332661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a12a8-723d-467a-9dff-e80f1e94f811 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:48.332953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a12a8-723d-467a-9dff-e80f1e94f811 None admin] PATH_INFO: `/v3/roles/0a56be0718ea495f8a241ae2447069cf` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:48.334381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a12a8-723d-467a-9dff-e80f1e94f811 None admin] RBAC: Authorizing `identity:get_role(role_id=0a56be0718ea495f8a241ae2447069cf)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:48.335778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a12a8-723d-467a-9dff-e80f1e94f811 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:48.337094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 566/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:25:48 2022] GET /identity/v3/roles/0a56be0718ea495f8a241ae2447069cf => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:48.342712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a79097-56f2-4f25-b0dd-9fff30d09d7b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:48.368832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:48.438239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a79097-56f2-4f25-b0dd-9fff30d09d7b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:48.470494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a5830-665c-4ed4-8aa7-333a5e38594f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:48.471135 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a5830-665c-4ed4-8aa7-333a5e38594f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:48.471416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a5830-665c-4ed4-8aa7-333a5e38594f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:48.471686 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a5830-665c-4ed4-8aa7-333a5e38594f None admin] PATH_INFO: `/v3/users/6a095323aea74c4dad75ac737736c648` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:48.472829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a5830-665c-4ed4-8aa7-333a5e38594f None admin] RBAC: Authorizing `identity:get_user(user_id=6a095323aea74c4dad75ac737736c648)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:48.474818 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a5830-665c-4ed4-8aa7-333a5e38594f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:48.476413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 561/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:25:48 2022] GET /identity/v3/users/6a095323aea74c4dad75ac737736c648 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:48.482033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a12a8-723d-467a-9dff-e80f1e94f811 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:48.501441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:48.578398 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a12a8-723d-467a-9dff-e80f1e94f811 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:48.610309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7e058-ca8d-40f6-9244-7602db1d0bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:48.610976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7e058-ca8d-40f6-9244-7602db1d0bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:48.611292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7e058-ca8d-40f6-9244-7602db1d0bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:48.611569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7e058-ca8d-40f6-9244-7602db1d0bb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:48.676515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7e058-ca8d-40f6-9244-7602db1d0bb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:48.677954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7e058-ca8d-40f6-9244-7602db1d0bb4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:48.695155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:48.772541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 567/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:25:48.778305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a5830-665c-4ed4-8aa7-333a5e38594f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:48.868057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:48.874866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a5830-665c-4ed4-8aa7-333a5e38594f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:48.908043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8910c-3e47-4ead-91cd-fe614dba8636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:48.908668 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8910c-3e47-4ead-91cd-fe614dba8636 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:48.908951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8910c-3e47-4ead-91cd-fe614dba8636 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:48.909227 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8910c-3e47-4ead-91cd-fe614dba8636 None admin] PATH_INFO: `/v3/projects/6b06f96780cf48ffa922d449b321afc0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:48.910364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8910c-3e47-4ead-91cd-fe614dba8636 None admin] RBAC: Authorizing `identity:get_project(project_id=6b06f96780cf48ffa922d449b321afc0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:48.911875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8910c-3e47-4ead-91cd-fe614dba8636 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:48.913527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 562/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:25:48 2022] GET /identity/v3/projects/6b06f96780cf48ffa922d449b321afc0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:48.919529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7e058-ca8d-40f6-9244-7602db1d0bb4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:49.017255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7e058-ca8d-40f6-9244-7602db1d0bb4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:49.033801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:49.050662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8de1a-d49f-4399-8cb8-e5f82c2e5356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:49.051298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8de1a-d49f-4399-8cb8-e5f82c2e5356 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:49.052173 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8de1a-d49f-4399-8cb8-e5f82c2e5356 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:49.052590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8de1a-d49f-4399-8cb8-e5f82c2e5356 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:49.053376 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8de1a-d49f-4399-8cb8-e5f82c2e5356 None admin] RBAC: Adding query filter params (role.id=0a56be0718ea495f8a241ae2447069cf, scope.project.id=6b06f96780cf48ffa922d449b321afc0, user.id=6a095323aea74c4dad75ac737736c648) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:49.053690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8de1a-d49f-4399-8cb8-e5f82c2e5356 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:49.055137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8de1a-d49f-4399-8cb8-e5f82c2e5356 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:49.060797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 568/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:25:48 2022] GET /identity/v3/role_assignments?user.id=6a095323aea74c4dad75ac737736c648&scope.project.id=6b06f96780cf48ffa922d449b321afc0&role.id=0a56be0718ea495f8a241ae2447069cf => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:51.102359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae083d-85e5-4dc1-b936-956532438a89 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:51.102859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae083d-85e5-4dc1-b936-956532438a89 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:51.102974 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae083d-85e5-4dc1-b936-956532438a89 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:51.104298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 563/1131] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:51.109524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fd82c-5abf-4542-9479-f0071b0426ab None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:51.109894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fd82c-5abf-4542-9479-f0071b0426ab None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:51.110082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fd82c-5abf-4542-9479-f0071b0426ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:51.128229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-af4fd82c-5abf-4542-9479-f0071b0426ab None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:51.130058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-af4fd82c-5abf-4542-9479-f0071b0426ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:51.194505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 569/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:51.216764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae083d-85e5-4dc1-b936-956532438a89 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:51.316532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae083d-85e5-4dc1-b936-956532438a89 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:51.349807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83c146-a9fd-44d5-9822-c0b57bb02f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:51.350432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83c146-a9fd-44d5-9822-c0b57bb02f2d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:51.350718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83c146-a9fd-44d5-9822-c0b57bb02f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:51.350996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83c146-a9fd-44d5-9822-c0b57bb02f2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:51.352164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83c146-a9fd-44d5-9822-c0b57bb02f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:51.353587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83c146-a9fd-44d5-9822-c0b57bb02f2d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:51.355197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 564/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:51.361009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fd82c-5abf-4542-9479-f0071b0426ab None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:51.457129 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fd82c-5abf-4542-9479-f0071b0426ab None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:51.470340 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:51.491972 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f720f73-4ac6-46b1-879c-943ee5053ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:51.492564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f720f73-4ac6-46b1-879c-943ee5053ba9 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:51.492849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f720f73-4ac6-46b1-879c-943ee5053ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:51.493132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f720f73-4ac6-46b1-879c-943ee5053ba9 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:51.493807 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f720f73-4ac6-46b1-879c-943ee5053ba9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:51.495258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f720f73-4ac6-46b1-879c-943ee5053ba9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:51.520922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 570/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:25:51 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:25:53.559285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f2d78-e4de-47f3-89ac-a3b4c2afb305 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:53.559868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f2d78-e4de-47f3-89ac-a3b4c2afb305 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:53.559982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f2d78-e4de-47f3-89ac-a3b4c2afb305 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:53.562509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 565/1135] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:53.568519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09574a-9695-4f93-8fa3-73b4f4c0b107 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:53.568951 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09574a-9695-4f93-8fa3-73b4f4c0b107 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:53.569053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09574a-9695-4f93-8fa3-73b4f4c0b107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:53.587055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6f09574a-9695-4f93-8fa3-73b4f4c0b107 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:53.588253 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6f09574a-9695-4f93-8fa3-73b4f4c0b107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:53.654281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 571/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:53.678882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f2d78-e4de-47f3-89ac-a3b4c2afb305 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:53.709580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:53.778846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f2d78-e4de-47f3-89ac-a3b4c2afb305 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:53.810932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa119f-8f10-4260-88a1-9d563f8b5c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:53.811537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa119f-8f10-4260-88a1-9d563f8b5c9f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:53.811824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa119f-8f10-4260-88a1-9d563f8b5c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:53.812104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa119f-8f10-4260-88a1-9d563f8b5c9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:53.813251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa119f-8f10-4260-88a1-9d563f8b5c9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:53.814677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa119f-8f10-4260-88a1-9d563f8b5c9f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:53.816289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 566/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:25:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:53.822479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09574a-9695-4f93-8fa3-73b4f4c0b107 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:53.913254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:53.919797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09574a-9695-4f93-8fa3-73b4f4c0b107 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:53.952847 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ae1d2-e9a5-4afd-975d-540962114e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:53.953461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ae1d2-e9a5-4afd-975d-540962114e81 None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:53.953741 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ae1d2-e9a5-4afd-975d-540962114e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:53.954014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ae1d2-e9a5-4afd-975d-540962114e81 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:53.954695 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ae1d2-e9a5-4afd-975d-540962114e81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:53.956159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ae1d2-e9a5-4afd-975d-540962114e81 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:53.977103 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 572/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:25:53 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:25:56.021104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37aaf4-62d7-41f4-acc8-f9e84177056a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:56.021630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37aaf4-62d7-41f4-acc8-f9e84177056a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:56.021729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37aaf4-62d7-41f4-acc8-f9e84177056a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:56.023040 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 567/1139] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:56.028234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79ee70-1df6-43ca-a9a5-16d85f6808b9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:56.028666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79ee70-1df6-43ca-a9a5-16d85f6808b9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:56.028798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79ee70-1df6-43ca-a9a5-16d85f6808b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:56.046454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5b79ee70-1df6-43ca-a9a5-16d85f6808b9 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:56.047670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5b79ee70-1df6-43ca-a9a5-16d85f6808b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:56.111752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 573/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:56.136125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37aaf4-62d7-41f4-acc8-f9e84177056a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:56.229537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:56.234531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37aaf4-62d7-41f4-acc8-f9e84177056a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:56.267736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7bd68-3e12-493a-aa81-3b18931f1d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:56.268362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7bd68-3e12-493a-aa81-3b18931f1d1f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:56.268649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7bd68-3e12-493a-aa81-3b18931f1d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:56.268926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7bd68-3e12-493a-aa81-3b18931f1d1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:56.279390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7bd68-3e12-493a-aa81-3b18931f1d1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-07a7bd68-3e12-493a-aa81-3b18931f1d1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:56.284247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:56.285386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:56.286343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 568/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:56.291634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79ee70-1df6-43ca-a9a5-16d85f6808b9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:56.372554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:56.400724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79ee70-1df6-43ca-a9a5-16d85f6808b9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:56.443834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822c8d9-18f6-4526-a198-93b73dd0d2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:56.444464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822c8d9-18f6-4526-a198-93b73dd0d2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:56.444748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822c8d9-18f6-4526-a198-93b73dd0d2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:56.445027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822c8d9-18f6-4526-a198-93b73dd0d2b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:56.445744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822c8d9-18f6-4526-a198-93b73dd0d2b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:56.446055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822c8d9-18f6-4526-a198-93b73dd0d2b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:56.447458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822c8d9-18f6-4526-a198-93b73dd0d2b8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:56.455784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 574/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:25:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:56.462152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7bd68-3e12-493a-aa81-3b18931f1d1f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:56.558203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7bd68-3e12-493a-aa81-3b18931f1d1f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:56.590742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f73d6-ebf1-4257-a8bc-568b62fea708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:56.591394 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f73d6-ebf1-4257-a8bc-568b62fea708 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:56.591681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f73d6-ebf1-4257-a8bc-568b62fea708 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:56.591962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f73d6-ebf1-4257-a8bc-568b62fea708 None admin] PATH_INFO: `/v3/users/ebbf2ce4515e430b95b8444f879322cd` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:56.605058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f73d6-ebf1-4257-a8bc-568b62fea708 None admin] RBAC: Authorizing `identity:get_user(user_id=ebbf2ce4515e430b95b8444f879322cd)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:56.606489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f73d6-ebf1-4257-a8bc-568b62fea708 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:56.608138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 569/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:25:56 2022] GET /identity/v3/users/ebbf2ce4515e430b95b8444f879322cd => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:56.614377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822c8d9-18f6-4526-a198-93b73dd0d2b8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:56.711914 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822c8d9-18f6-4526-a198-93b73dd0d2b8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:56.744348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e5c34-4df3-43dd-a05d-ebbe7608df8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:56.744949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e5c34-4df3-43dd-a05d-ebbe7608df8e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:56.745235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e5c34-4df3-43dd-a05d-ebbe7608df8e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:56.745509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e5c34-4df3-43dd-a05d-ebbe7608df8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:56.756822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:56.810330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e5c34-4df3-43dd-a05d-ebbe7608df8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:56.811742 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e5c34-4df3-43dd-a05d-ebbe7608df8e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:56.905180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 575/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:25:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:25:56.911822 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f73d6-ebf1-4257-a8bc-568b62fea708 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:56.930107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:57.011746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f73d6-ebf1-4257-a8bc-568b62fea708 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:57.044918 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975238b-8804-4cc9-9186-942f5cbe507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:57.045546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975238b-8804-4cc9-9186-942f5cbe507a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:57.045830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975238b-8804-4cc9-9186-942f5cbe507a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:57.046106 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975238b-8804-4cc9-9186-942f5cbe507a None admin] PATH_INFO: `/v3/projects/400fd53c96ed4890bcd450c7636c5ef0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:57.047266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975238b-8804-4cc9-9186-942f5cbe507a None admin] RBAC: Authorizing `identity:get_project(project_id=400fd53c96ed4890bcd450c7636c5ef0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:57.048689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975238b-8804-4cc9-9186-942f5cbe507a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:57.050337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 570/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:25:56 2022] GET /identity/v3/projects/400fd53c96ed4890bcd450c7636c5ef0 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:57.057092 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e5c34-4df3-43dd-a05d-ebbe7608df8e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:57.098370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:57.152408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e5c34-4df3-43dd-a05d-ebbe7608df8e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:57.184748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eaf6b-4cbf-4a51-9e2c-69116bfe873c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:57.185338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eaf6b-4cbf-4a51-9e2c-69116bfe873c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:57.185618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eaf6b-4cbf-4a51-9e2c-69116bfe873c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:57.185888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eaf6b-4cbf-4a51-9e2c-69116bfe873c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:57.186669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eaf6b-4cbf-4a51-9e2c-69116bfe873c None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=400fd53c96ed4890bcd450c7636c5ef0, user.id=ebbf2ce4515e430b95b8444f879322cd) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:57.186977 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eaf6b-4cbf-4a51-9e2c-69116bfe873c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:57.188416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eaf6b-4cbf-4a51-9e2c-69116bfe873c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:57.194200 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 576/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:25:57 2022] GET /identity/v3/role_assignments?user.id=ebbf2ce4515e430b95b8444f879322cd&scope.project.id=400fd53c96ed4890bcd450c7636c5ef0&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:59.158769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e6d9e-ed7a-41eb-b542-e5d94a808ac0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:59.159374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e6d9e-ed7a-41eb-b542-e5d94a808ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:59.159374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e6d9e-ed7a-41eb-b542-e5d94a808ac0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:59.160874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 571/1147] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:25:59.166006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773276dc-60c6-48f7-adcb-39c04a012370 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:59.166428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773276dc-60c6-48f7-adcb-39c04a012370 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:59.166570 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773276dc-60c6-48f7-adcb-39c04a012370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:59.184082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-773276dc-60c6-48f7-adcb-39c04a012370 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:25:59.185263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-773276dc-60c6-48f7-adcb-39c04a012370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:25:59.249690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 577/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:25:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:25:59.272625 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e6d9e-ed7a-41eb-b542-e5d94a808ac0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:59.357928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:59.374835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e6d9e-ed7a-41eb-b542-e5d94a808ac0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:59.407982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1207ae-1f7a-448c-aaad-7ba135eedf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:59.408620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1207ae-1f7a-448c-aaad-7ba135eedf34 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:59.408905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1207ae-1f7a-448c-aaad-7ba135eedf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:59.409185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1207ae-1f7a-448c-aaad-7ba135eedf34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:59.419019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1207ae-1f7a-448c-aaad-7ba135eedf34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4c1207ae-1f7a-448c-aaad-7ba135eedf34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:25:59.423893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:25:59.424978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:25:59.426731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 572/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:25:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:25:59.431808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773276dc-60c6-48f7-adcb-39c04a012370 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:59.465421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:59.529614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773276dc-60c6-48f7-adcb-39c04a012370 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:59.562627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c6f33-9231-4abd-a865-814bbb418a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:59.563262 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c6f33-9231-4abd-a865-814bbb418a78 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:59.563550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c6f33-9231-4abd-a865-814bbb418a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:59.563830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c6f33-9231-4abd-a865-814bbb418a78 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:59.564538 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c6f33-9231-4abd-a865-814bbb418a78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:25:59.564849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c6f33-9231-4abd-a865-814bbb418a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:59.566204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c6f33-9231-4abd-a865-814bbb418a78 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:59.575007 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 578/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:25:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:59.580681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1207ae-1f7a-448c-aaad-7ba135eedf34 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:59.679206 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1207ae-1f7a-448c-aaad-7ba135eedf34 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:59.711726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e0e58-26d6-4ace-befe-be00843afc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:59.712342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e0e58-26d6-4ace-befe-be00843afc49 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:59.712627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e0e58-26d6-4ace-befe-be00843afc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:59.712903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e0e58-26d6-4ace-befe-be00843afc49 None admin] PATH_INFO: `/v3/users/ebbf2ce4515e430b95b8444f879322cd` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:59.714051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e0e58-26d6-4ace-befe-be00843afc49 None admin] RBAC: Authorizing `identity:get_user(user_id=ebbf2ce4515e430b95b8444f879322cd)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:59.715508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e0e58-26d6-4ace-befe-be00843afc49 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:59.717090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 573/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:25:59 2022] GET /identity/v3/users/ebbf2ce4515e430b95b8444f879322cd => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:59.723380 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c6f33-9231-4abd-a865-814bbb418a78 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:59.817541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:59.821491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c6f33-9231-4abd-a865-814bbb418a78 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:59.853877 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9974a-0943-4d5c-b4d0-e625a816d799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:59.854499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9974a-0943-4d5c-b4d0-e625a816d799 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:59.854781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9974a-0943-4d5c-b4d0-e625a816d799 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:59.855053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9974a-0943-4d5c-b4d0-e625a816d799 None admin] PATH_INFO: `/v3/projects/400fd53c96ed4890bcd450c7636c5ef0` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:59.856209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9974a-0943-4d5c-b4d0-e625a816d799 None admin] RBAC: Authorizing `identity:get_project(project_id=400fd53c96ed4890bcd450c7636c5ef0)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:25:59.857613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9974a-0943-4d5c-b4d0-e625a816d799 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:25:59.859269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 579/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:25:59 2022] GET /identity/v3/projects/400fd53c96ed4890bcd450c7636c5ef0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:25:59.865696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e0e58-26d6-4ace-befe-be00843afc49 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:25:59.880122 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:25:59.963001 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e0e58-26d6-4ace-befe-be00843afc49 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:25:59.996755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c446a-3550-4f86-b972-828d955491d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:25:59.997409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c446a-3550-4f86-b972-828d955491d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:25:59.997694 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c446a-3550-4f86-b972-828d955491d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:25:59.997974 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c446a-3550-4f86-b972-828d955491d2 None admin] PATH_INFO: `/v3/projects/400fd53c96ed4890bcd450c7636c5ef0/users/ebbf2ce4515e430b95b8444f879322cd/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:25:59.999923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c446a-3550-4f86-b972-828d955491d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=400fd53c96ed4890bcd450c7636c5ef0, user_id=ebbf2ce4515e430b95b8444f879322cd, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:00.001350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c446a-3550-4f86-b972-828d955491d2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:00.014728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 574/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:25:59 2022] PUT /identity/v3/projects/400fd53c96ed4890bcd450c7636c5ef0/users/ebbf2ce4515e430b95b8444f879322cd/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:26:01.960705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce859a1-5899-482d-bc53-f3b4dca3956a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:01.961227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce859a1-5899-482d-bc53-f3b4dca3956a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:01.961348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce859a1-5899-482d-bc53-f3b4dca3956a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:01.962610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 580/1154] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:26:01.967743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c8b04-e2b6-44b6-8653-b040553631c3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:01.968144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c8b04-e2b6-44b6-8653-b040553631c3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:01.968299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c8b04-e2b6-44b6-8653-b040553631c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:01.986654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4e5c8b04-e2b6-44b6-8653-b040553631c3 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:26:01.987866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4e5c8b04-e2b6-44b6-8653-b040553631c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:26:02.053410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 575/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:26:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:26:02.075879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce859a1-5899-482d-bc53-f3b4dca3956a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:02.174056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce859a1-5899-482d-bc53-f3b4dca3956a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:02.206943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b501ab0-aef9-44c3-9dd8-920c54652a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:02.207669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b501ab0-aef9-44c3-9dd8-920c54652a81 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:02.207952 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b501ab0-aef9-44c3-9dd8-920c54652a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:02.208228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b501ab0-aef9-44c3-9dd8-920c54652a81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:02.218013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b501ab0-aef9-44c3-9dd8-920c54652a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-4b501ab0-aef9-44c3-9dd8-920c54652a81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:26:02.223352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:26:02.224463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:26:02.225516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 581/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:26:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:26:02.230344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c8b04-e2b6-44b6-8653-b040553631c3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:02.232541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:02.329272 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c8b04-e2b6-44b6-8653-b040553631c3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:02.367169 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bde83-7171-409e-82bb-850162f124bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:02.367777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bde83-7171-409e-82bb-850162f124bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:02.368059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bde83-7171-409e-82bb-850162f124bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:02.368337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bde83-7171-409e-82bb-850162f124bb None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:02.369045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bde83-7171-409e-82bb-850162f124bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:26:02.369352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bde83-7171-409e-82bb-850162f124bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:02.370713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bde83-7171-409e-82bb-850162f124bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:02.378504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 576/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:26:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:02.385095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b501ab0-aef9-44c3-9dd8-920c54652a81 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:02.412180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:02.483346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b501ab0-aef9-44c3-9dd8-920c54652a81 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:02.517077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47425ae0-a9c9-4039-9cea-ddefa4a4605d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:02.517698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47425ae0-a9c9-4039-9cea-ddefa4a4605d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:02.517982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47425ae0-a9c9-4039-9cea-ddefa4a4605d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:02.518261 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47425ae0-a9c9-4039-9cea-ddefa4a4605d None admin] PATH_INFO: `/v3/users/ebbf2ce4515e430b95b8444f879322cd` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:02.519781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47425ae0-a9c9-4039-9cea-ddefa4a4605d None admin] RBAC: Authorizing `identity:get_user(user_id=ebbf2ce4515e430b95b8444f879322cd)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:02.521282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47425ae0-a9c9-4039-9cea-ddefa4a4605d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:02.522870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 582/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:26:02 2022] GET /identity/v3/users/ebbf2ce4515e430b95b8444f879322cd => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:02.528429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bde83-7171-409e-82bb-850162f124bb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:02.589494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:02.626254 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bde83-7171-409e-82bb-850162f124bb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:02.659360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fac55-104d-411c-9440-b2b192256818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:02.660542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fac55-104d-411c-9440-b2b192256818 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:02.660828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fac55-104d-411c-9440-b2b192256818 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:02.661100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fac55-104d-411c-9440-b2b192256818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:02.724472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fac55-104d-411c-9440-b2b192256818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:02.725898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fac55-104d-411c-9440-b2b192256818 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:02.782292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:02.819304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 577/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:26:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:26:02.825125 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47425ae0-a9c9-4039-9cea-ddefa4a4605d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:02.916773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:02.923956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47425ae0-a9c9-4039-9cea-ddefa4a4605d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:02.957083 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be921ff5-c300-46ed-afca-188af7891cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:02.957938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be921ff5-c300-46ed-afca-188af7891cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:02.958225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be921ff5-c300-46ed-afca-188af7891cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:02.958508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be921ff5-c300-46ed-afca-188af7891cc2 None admin] PATH_INFO: `/v3/projects/400fd53c96ed4890bcd450c7636c5ef0` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:02.959688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be921ff5-c300-46ed-afca-188af7891cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=400fd53c96ed4890bcd450c7636c5ef0)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:02.961118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be921ff5-c300-46ed-afca-188af7891cc2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:02.962790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 583/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:26:02 2022] GET /identity/v3/projects/400fd53c96ed4890bcd450c7636c5ef0 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:02.969005 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fac55-104d-411c-9440-b2b192256818 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:03.065306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fac55-104d-411c-9440-b2b192256818 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:03.098280 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b5989-b312-4135-a13e-9cf52b7f3c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:03.098874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b5989-b312-4135-a13e-9cf52b7f3c81 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:03.099242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b5989-b312-4135-a13e-9cf52b7f3c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:03.099521 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b5989-b312-4135-a13e-9cf52b7f3c81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:03.100295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b5989-b312-4135-a13e-9cf52b7f3c81 None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=400fd53c96ed4890bcd450c7636c5ef0, user.id=ebbf2ce4515e430b95b8444f879322cd) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:26:03.100601 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b5989-b312-4135-a13e-9cf52b7f3c81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:03.102011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b5989-b312-4135-a13e-9cf52b7f3c81 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:03.107662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 578/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:26:02 2022] GET /identity/v3/role_assignments?user.id=ebbf2ce4515e430b95b8444f879322cd&scope.project.id=400fd53c96ed4890bcd450c7636c5ef0&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:05.254375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6998df-e082-4ddd-b1fc-78d70854c7a0 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:05.254873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6998df-e082-4ddd-b1fc-78d70854c7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:05.254873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6998df-e082-4ddd-b1fc-78d70854c7a0 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:05.256336 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 584/1162] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:26:05.261756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9378c4-84ae-4ea2-87b4-8a68b7a96d95 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:05.262167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9378c4-84ae-4ea2-87b4-8a68b7a96d95 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:05.262308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9378c4-84ae-4ea2-87b4-8a68b7a96d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:05.280817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1b9378c4-84ae-4ea2-87b4-8a68b7a96d95 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:26:05.281993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1b9378c4-84ae-4ea2-87b4-8a68b7a96d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:26:05.304207 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:05.347612 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 579/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:26:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:26:05.369740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6998df-e082-4ddd-b1fc-78d70854c7a0 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:05.475840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6998df-e082-4ddd-b1fc-78d70854c7a0 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:05.508703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97489019-3677-4953-a2c0-e08d1996c77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:05.509258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97489019-3677-4953-a2c0-e08d1996c77c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:05.509547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97489019-3677-4953-a2c0-e08d1996c77c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:05.509829 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97489019-3677-4953-a2c0-e08d1996c77c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:05.539266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:05.575597 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97489019-3677-4953-a2c0-e08d1996c77c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:05.577058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97489019-3677-4953-a2c0-e08d1996c77c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:05.674647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 585/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:26:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:26:05.681564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9378c4-84ae-4ea2-87b4-8a68b7a96d95 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:05.777109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9378c4-84ae-4ea2-87b4-8a68b7a96d95 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:05.810839 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cd045-900e-4155-9cc9-276fa295a80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:05.811501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cd045-900e-4155-9cc9-276fa295a80b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:05.811789 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cd045-900e-4155-9cc9-276fa295a80b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:05.812069 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cd045-900e-4155-9cc9-276fa295a80b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:05.818705 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cd045-900e-4155-9cc9-276fa295a80b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:05.820165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cd045-900e-4155-9cc9-276fa295a80b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f86cd045-900e-4155-9cc9-276fa295a80b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:26:05.831357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:26:05.832771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:26:05.833970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 580/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Aug 1 07:26:05 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:26:05.838460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97489019-3677-4953-a2c0-e08d1996c77c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:05.897781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:05.935152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97489019-3677-4953-a2c0-e08d1996c77c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:05.967451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f1751-01f5-41b3-b423-0fe2944c7658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:05.968064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f1751-01f5-41b3-b423-0fe2944c7658 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:05.968345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f1751-01f5-41b3-b423-0fe2944c7658 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:05.968621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f1751-01f5-41b3-b423-0fe2944c7658 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:05.969303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f1751-01f5-41b3-b423-0fe2944c7658 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:26:05.969607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f1751-01f5-41b3-b423-0fe2944c7658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:05.970969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f1751-01f5-41b3-b423-0fe2944c7658 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:05.978657 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 586/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Aug 1 07:26:05 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:07.958575 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d1181-bd47-40f6-9c05-2df07d141810 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:07.959140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d1181-bd47-40f6-9c05-2df07d141810 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:07.959140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d1181-bd47-40f6-9c05-2df07d141810 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:07.960514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 581/1167] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:26:07.965767 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91632be-7acb-4d37-b3fd-0039e4c8c20b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:07.966159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91632be-7acb-4d37-b3fd-0039e4c8c20b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:07.966336 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91632be-7acb-4d37-b3fd-0039e4c8c20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:07.984660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d91632be-7acb-4d37-b3fd-0039e4c8c20b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:26:07.985852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d91632be-7acb-4d37-b3fd-0039e4c8c20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:26:08.050952 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 587/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:26:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:26:08.072919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d1181-bd47-40f6-9c05-2df07d141810 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:08.091604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:08.173911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d1181-bd47-40f6-9c05-2df07d141810 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:08.207182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c2e55-a606-405b-9ba2-f475836bcbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:08.207782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c2e55-a606-405b-9ba2-f475836bcbbe None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:08.208072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c2e55-a606-405b-9ba2-f475836bcbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:08.208344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c2e55-a606-405b-9ba2-f475836bcbbe None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:08.209107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c2e55-a606-405b-9ba2-f475836bcbbe None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:08.210412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c2e55-a606-405b-9ba2-f475836bcbbe None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:08.226200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 582/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Mon Aug 1 07:26:08 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:26:10.250441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d7bc0-8388-4aa4-a0b8-a1bb8ea15024 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:10.250936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d7bc0-8388-4aa4-a0b8-a1bb8ea15024 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:10.250936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d7bc0-8388-4aa4-a0b8-a1bb8ea15024 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:10.252315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 588/1170] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:26:10.258466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49cff2-9537-416a-a1fc-97e847444400 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:10.258896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49cff2-9537-416a-a1fc-97e847444400 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:10.259107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49cff2-9537-416a-a1fc-97e847444400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:10.277473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3a49cff2-9537-416a-a1fc-97e847444400 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:26:10.278645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3a49cff2-9537-416a-a1fc-97e847444400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:26:10.342734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 583/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:26:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:26:10.365564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d7bc0-8388-4aa4-a0b8-a1bb8ea15024 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:10.422648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:10.466984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d7bc0-8388-4aa4-a0b8-a1bb8ea15024 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:10.499735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace78486-011b-4746-8798-251af3844349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:10.500361 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace78486-011b-4746-8798-251af3844349 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:10.500699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace78486-011b-4746-8798-251af3844349 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:10.500930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace78486-011b-4746-8798-251af3844349 None admin] PATH_INFO: `/v3/domains/22d3b09ebf054b24b6986476eb465e26` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:10.512265 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace78486-011b-4746-8798-251af3844349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=22d3b09ebf054b24b6986476eb465e26)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:10.513689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace78486-011b-4746-8798-251af3844349 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:10.515305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 589/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Mon Aug 1 07:26:10 2022] GET /identity/v3/domains/22d3b09ebf054b24b6986476eb465e26 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:10.521671 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49cff2-9537-416a-a1fc-97e847444400 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:10.531494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:10.616978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49cff2-9537-416a-a1fc-97e847444400 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:10.650940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13fd21-27db-4e77-b705-75af4d3ed33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:10.651576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13fd21-27db-4e77-b705-75af4d3ed33e None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:10.651856 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13fd21-27db-4e77-b705-75af4d3ed33e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:10.652130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13fd21-27db-4e77-b705-75af4d3ed33e None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:10.652802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13fd21-27db-4e77-b705-75af4d3ed33e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:10.654216 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13fd21-27db-4e77-b705-75af4d3ed33e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:10.683664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 584/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:26:10 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:26:12.699884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7940b77-477b-4666-b780-4dcf5fe0f18a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:12.700433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7940b77-477b-4666-b780-4dcf5fe0f18a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:12.700433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7940b77-477b-4666-b780-4dcf5fe0f18a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:12.701783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 590/1174] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:26:12.707003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f3adc-ab00-4dca-b24d-28b86b49c6ae None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:12.707418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f3adc-ab00-4dca-b24d-28b86b49c6ae None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:12.707585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f3adc-ab00-4dca-b24d-28b86b49c6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:12.726077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a96f3adc-ab00-4dca-b24d-28b86b49c6ae None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:26:12.727307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a96f3adc-ab00-4dca-b24d-28b86b49c6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:26:12.782186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:12.794796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 585/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:26:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:26:12.819235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7940b77-477b-4666-b780-4dcf5fe0f18a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:12.918023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7940b77-477b-4666-b780-4dcf5fe0f18a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:12.933461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:12.950464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3fa8f-6c9b-4dac-a226-67c98bc68d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:12.951036 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3fa8f-6c9b-4dac-a226-67c98bc68d51 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:12.951339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3fa8f-6c9b-4dac-a226-67c98bc68d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:12.951612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3fa8f-6c9b-4dac-a226-67c98bc68d51 None admin] PATH_INFO: `/v3/domains/22d3b09ebf054b24b6986476eb465e26` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:12.952743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3fa8f-6c9b-4dac-a226-67c98bc68d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=22d3b09ebf054b24b6986476eb465e26)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:12.954165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3fa8f-6c9b-4dac-a226-67c98bc68d51 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:12.955762 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 591/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Mon Aug 1 07:26:12 2022] GET /identity/v3/domains/22d3b09ebf054b24b6986476eb465e26 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:12.962168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f3adc-ab00-4dca-b24d-28b86b49c6ae None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:13.059231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f3adc-ab00-4dca-b24d-28b86b49c6ae None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:13.091755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78f54a-2ccc-4544-9ff3-d730dcc72ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:13.092365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78f54a-2ccc-4544-9ff3-d730dcc72ae7 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:13.092649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78f54a-2ccc-4544-9ff3-d730dcc72ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:13.092927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78f54a-2ccc-4544-9ff3-d730dcc72ae7 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:13.093597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78f54a-2ccc-4544-9ff3-d730dcc72ae7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:13.095021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78f54a-2ccc-4544-9ff3-d730dcc72ae7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:13.116712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 586/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:26:12 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:26:15.193810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3f167-d931-4028-a013-96c3f250f235 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:15.194381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3f167-d931-4028-a013-96c3f250f235 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:15.194381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3f167-d931-4028-a013-96c3f250f235 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:15.195751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 592/1178] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:26:15.201034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81da060-d69c-422e-964e-16e5d9300def None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:15.201460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81da060-d69c-422e-964e-16e5d9300def None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:15.201590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81da060-d69c-422e-964e-16e5d9300def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:15.220977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a81da060-d69c-422e-964e-16e5d9300def None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:26:15.222927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a81da060-d69c-422e-964e-16e5d9300def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:26:15.263009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:15.289147 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 587/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:26:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:26:15.311930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3f167-d931-4028-a013-96c3f250f235 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:15.409509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3f167-d931-4028-a013-96c3f250f235 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:15.442133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d8bf9-2b9e-4e32-a38e-8525c0b6b338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:15.442752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d8bf9-2b9e-4e32-a38e-8525c0b6b338 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:15.443033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d8bf9-2b9e-4e32-a38e-8525c0b6b338 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:15.443339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d8bf9-2b9e-4e32-a38e-8525c0b6b338 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:15.452909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d8bf9-2b9e-4e32-a38e-8525c0b6b338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-682d8bf9-2b9e-4e32-a38e-8525c0b6b338 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:26:15.458390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:26:15.459605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:26:15.460652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 593/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:26:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:26:15.465439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81da060-d69c-422e-964e-16e5d9300def None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:15.566945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81da060-d69c-422e-964e-16e5d9300def None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:15.599569 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca765b-ef29-40ac-af84-2ed6631637ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:15.600174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca765b-ef29-40ac-af84-2ed6631637ea None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:15.600457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca765b-ef29-40ac-af84-2ed6631637ea None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:15.600734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca765b-ef29-40ac-af84-2ed6631637ea None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:15.601437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca765b-ef29-40ac-af84-2ed6631637ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:26:15.601742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca765b-ef29-40ac-af84-2ed6631637ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:15.603113 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca765b-ef29-40ac-af84-2ed6631637ea None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:15.610846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 588/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:26:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:15.616953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d8bf9-2b9e-4e32-a38e-8525c0b6b338 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:15.665365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:15.717731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d8bf9-2b9e-4e32-a38e-8525c0b6b338 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:15.749450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4268370-6eb1-44cf-a0ec-4a7c32a4fc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:15.750076 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4268370-6eb1-44cf-a0ec-4a7c32a4fc8d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:15.750356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4268370-6eb1-44cf-a0ec-4a7c32a4fc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:15.750629 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4268370-6eb1-44cf-a0ec-4a7c32a4fc8d None admin] PATH_INFO: `/v3/users/c722c532efda462daf9bf110df255953` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:15.763880 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4268370-6eb1-44cf-a0ec-4a7c32a4fc8d None admin] RBAC: Authorizing `identity:get_user(user_id=c722c532efda462daf9bf110df255953)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:15.765306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4268370-6eb1-44cf-a0ec-4a7c32a4fc8d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:15.766926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 594/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:26:15 2022] GET /identity/v3/users/c722c532efda462daf9bf110df255953 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:15.773391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca765b-ef29-40ac-af84-2ed6631637ea None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:15.811902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:15.869594 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca765b-ef29-40ac-af84-2ed6631637ea None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:15.902238 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca90db5-ee5a-4717-83e2-f536a89f0e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:15.902837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca90db5-ee5a-4717-83e2-f536a89f0e0b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:15.903146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca90db5-ee5a-4717-83e2-f536a89f0e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:15.903422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca90db5-ee5a-4717-83e2-f536a89f0e0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:15.967422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca90db5-ee5a-4717-83e2-f536a89f0e0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:15.968869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca90db5-ee5a-4717-83e2-f536a89f0e0b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:16.005641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:16.063804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 589/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:26:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:26:16.070244 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4268370-6eb1-44cf-a0ec-4a7c32a4fc8d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:16.166887 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4268370-6eb1-44cf-a0ec-4a7c32a4fc8d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:16.168352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:16.200745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c092ec-fb19-4511-8547-a9a5feacf5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:16.201374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c092ec-fb19-4511-8547-a9a5feacf5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:16.201661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c092ec-fb19-4511-8547-a9a5feacf5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:16.201939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c092ec-fb19-4511-8547-a9a5feacf5f3 None admin] PATH_INFO: `/v3/projects/16e919bbf5e74516a7d6ca748edcf419` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:16.203114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c092ec-fb19-4511-8547-a9a5feacf5f3 None admin] RBAC: Authorizing `identity:get_project(project_id=16e919bbf5e74516a7d6ca748edcf419)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:16.204535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c092ec-fb19-4511-8547-a9a5feacf5f3 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:16.206784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 595/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:26:16 2022] GET /identity/v3/projects/16e919bbf5e74516a7d6ca748edcf419 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:16.212796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca90db5-ee5a-4717-83e2-f536a89f0e0b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:16.309256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca90db5-ee5a-4717-83e2-f536a89f0e0b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:16.343281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd4c01-98a0-4a97-a050-52d49b9ebe18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:16.343881 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd4c01-98a0-4a97-a050-52d49b9ebe18 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:16.344163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd4c01-98a0-4a97-a050-52d49b9ebe18 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:16.344440 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd4c01-98a0-4a97-a050-52d49b9ebe18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:16.345215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd4c01-98a0-4a97-a050-52d49b9ebe18 None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=16e919bbf5e74516a7d6ca748edcf419, user.id=c722c532efda462daf9bf110df255953) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:26:16.345526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd4c01-98a0-4a97-a050-52d49b9ebe18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:16.347265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd4c01-98a0-4a97-a050-52d49b9ebe18 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:16.352572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 590/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:26:16 2022] GET /identity/v3/role_assignments?user.id=c722c532efda462daf9bf110df255953&scope.project.id=16e919bbf5e74516a7d6ca748edcf419&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:18.342937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb47133-1cca-4c1b-aa22-5b7786021335 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:18.343476 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb47133-1cca-4c1b-aa22-5b7786021335 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:18.343476 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb47133-1cca-4c1b-aa22-5b7786021335 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:18.344794 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 596/1186] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:26:18.350776 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732199fa-35d4-4c7c-b8cf-816bc70bbcb0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:18.351191 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732199fa-35d4-4c7c-b8cf-816bc70bbcb0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:18.351369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732199fa-35d4-4c7c-b8cf-816bc70bbcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:18.357652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:18.370054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-732199fa-35d4-4c7c-b8cf-816bc70bbcb0 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:26:18.371270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-732199fa-35d4-4c7c-b8cf-816bc70bbcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:26:18.435464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 591/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:26:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:26:18.458161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb47133-1cca-4c1b-aa22-5b7786021335 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:18.556567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb47133-1cca-4c1b-aa22-5b7786021335 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:18.589928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae2182-216d-4bd5-9ddc-7753baca1f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:18.590550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae2182-216d-4bd5-9ddc-7753baca1f54 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:18.590832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae2182-216d-4bd5-9ddc-7753baca1f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:18.591133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae2182-216d-4bd5-9ddc-7753baca1f54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:18.600952 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae2182-216d-4bd5-9ddc-7753baca1f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-8bae2182-216d-4bd5-9ddc-7753baca1f54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:26:18.605673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:26:18.606726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:26:18.607785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 597/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:26:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:26:18.613136 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732199fa-35d4-4c7c-b8cf-816bc70bbcb0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:18.708248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732199fa-35d4-4c7c-b8cf-816bc70bbcb0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:18.735642 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:18.741104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94be962-9e8d-438c-bffd-9bdb5fcd6169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:18.741712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94be962-9e8d-438c-bffd-9bdb5fcd6169 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:18.741994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94be962-9e8d-438c-bffd-9bdb5fcd6169 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:18.742276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94be962-9e8d-438c-bffd-9bdb5fcd6169 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:18.742982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94be962-9e8d-438c-bffd-9bdb5fcd6169 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:26:18.743319 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94be962-9e8d-438c-bffd-9bdb5fcd6169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:18.744669 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94be962-9e8d-438c-bffd-9bdb5fcd6169 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:18.752483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 592/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:26:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:18.759039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae2182-216d-4bd5-9ddc-7753baca1f54 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:18.789118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:18.855736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae2182-216d-4bd5-9ddc-7753baca1f54 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:18.889443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c41cf8-bb96-425f-b2a6-9be1546932d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:18.890071 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c41cf8-bb96-425f-b2a6-9be1546932d4 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:18.890353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c41cf8-bb96-425f-b2a6-9be1546932d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:18.890627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c41cf8-bb96-425f-b2a6-9be1546932d4 None admin] PATH_INFO: `/v3/users/c722c532efda462daf9bf110df255953` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:18.891783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c41cf8-bb96-425f-b2a6-9be1546932d4 None admin] RBAC: Authorizing `identity:get_user(user_id=c722c532efda462daf9bf110df255953)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:18.893253 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c41cf8-bb96-425f-b2a6-9be1546932d4 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:18.894834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 598/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:26:18 2022] GET /identity/v3/users/c722c532efda462daf9bf110df255953 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:18.900459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94be962-9e8d-438c-bffd-9bdb5fcd6169 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:18.996071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94be962-9e8d-438c-bffd-9bdb5fcd6169 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:19.030646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b6c78-fe56-43cb-9467-1f65f7d916b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:19.031311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b6c78-fe56-43cb-9467-1f65f7d916b4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:19.031597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b6c78-fe56-43cb-9467-1f65f7d916b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:19.031874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b6c78-fe56-43cb-9467-1f65f7d916b4 None admin] PATH_INFO: `/v3/projects/16e919bbf5e74516a7d6ca748edcf419` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:19.033014 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b6c78-fe56-43cb-9467-1f65f7d916b4 None admin] RBAC: Authorizing `identity:get_project(project_id=16e919bbf5e74516a7d6ca748edcf419)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:19.035353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b6c78-fe56-43cb-9467-1f65f7d916b4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:19.037694 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 593/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:26:18 2022] GET /identity/v3/projects/16e919bbf5e74516a7d6ca748edcf419 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:19.043880 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c41cf8-bb96-425f-b2a6-9be1546932d4 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:19.136345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:19.140273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c41cf8-bb96-425f-b2a6-9be1546932d4 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:19.173475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af86c49-0911-429e-be3b-e79df4e69133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:19.174122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af86c49-0911-429e-be3b-e79df4e69133 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:19.174431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af86c49-0911-429e-be3b-e79df4e69133 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:19.174710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af86c49-0911-429e-be3b-e79df4e69133 None admin] PATH_INFO: `/v3/projects/16e919bbf5e74516a7d6ca748edcf419/users/c722c532efda462daf9bf110df255953/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:19.176786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af86c49-0911-429e-be3b-e79df4e69133 None admin] RBAC: Authorizing `identity:create_grant(project_id=16e919bbf5e74516a7d6ca748edcf419, user_id=c722c532efda462daf9bf110df255953, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:19.178217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af86c49-0911-429e-be3b-e79df4e69133 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:19.190879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 599/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:26:19 2022] PUT /identity/v3/projects/16e919bbf5e74516a7d6ca748edcf419/users/c722c532efda462daf9bf110df255953/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:26:21.168866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec83fa-74ed-428d-9a6c-d50b05fdd802 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:21.169419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec83fa-74ed-428d-9a6c-d50b05fdd802 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:21.169419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec83fa-74ed-428d-9a6c-d50b05fdd802 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:21.170774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 594/1193] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:26:21.176398 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c834888-b74d-4883-a2f8-ef8acdd2e512 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:21.176791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c834888-b74d-4883-a2f8-ef8acdd2e512 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:21.176959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c834888-b74d-4883-a2f8-ef8acdd2e512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:21.195837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1c834888-b74d-4883-a2f8-ef8acdd2e512 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:26:21.197028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1c834888-b74d-4883-a2f8-ef8acdd2e512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:26:21.262047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 600/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:26:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:26:21.284805 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec83fa-74ed-428d-9a6c-d50b05fdd802 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:21.345021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:21.384492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec83fa-74ed-428d-9a6c-d50b05fdd802 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:21.417174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f5f40-3801-4c15-932c-9a969373c0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:21.417799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f5f40-3801-4c15-932c-9a969373c0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:21.418087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f5f40-3801-4c15-932c-9a969373c0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:21.418363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f5f40-3801-4c15-932c-9a969373c0e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:21.429695 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f5f40-3801-4c15-932c-9a969373c0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-505f5f40-3801-4c15-932c-9a969373c0e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:26:21.434526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:26:21.435649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:26:21.436674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 595/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:26:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:26:21.441724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c834888-b74d-4883-a2f8-ef8acdd2e512 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:21.538799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:21.552429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c834888-b74d-4883-a2f8-ef8acdd2e512 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:21.584927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcb200-21d2-463b-a009-283d9f88a05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:21.585534 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcb200-21d2-463b-a009-283d9f88a05a None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:21.585818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcb200-21d2-463b-a009-283d9f88a05a None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:21.586093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcb200-21d2-463b-a009-283d9f88a05a None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:21.586799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcb200-21d2-463b-a009-283d9f88a05a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:26:21.587155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcb200-21d2-463b-a009-283d9f88a05a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:21.588521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcb200-21d2-463b-a009-283d9f88a05a None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:21.597071 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 601/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:26:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:21.602687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f5f40-3801-4c15-932c-9a969373c0e2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:21.699485 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f5f40-3801-4c15-932c-9a969373c0e2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:21.720865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:21.733415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e62d8-bc86-4ff8-9121-0b1898d1bb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:21.734034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e62d8-bc86-4ff8-9121-0b1898d1bb96 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:21.734315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e62d8-bc86-4ff8-9121-0b1898d1bb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:21.734589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e62d8-bc86-4ff8-9121-0b1898d1bb96 None admin] PATH_INFO: `/v3/users/c722c532efda462daf9bf110df255953` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:21.735737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e62d8-bc86-4ff8-9121-0b1898d1bb96 None admin] RBAC: Authorizing `identity:get_user(user_id=c722c532efda462daf9bf110df255953)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:21.737902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e62d8-bc86-4ff8-9121-0b1898d1bb96 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:21.739441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 596/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Aug 1 07:26:21 2022] GET /identity/v3/users/c722c532efda462daf9bf110df255953 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:21.745184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcb200-21d2-463b-a009-283d9f88a05a None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:21.841207 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcb200-21d2-463b-a009-283d9f88a05a None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:21.873417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c61080-8722-421d-9979-81c266da6d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:21.874015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c61080-8722-421d-9979-81c266da6d56 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:21.874292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c61080-8722-421d-9979-81c266da6d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:21.874566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c61080-8722-421d-9979-81c266da6d56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:21.892531 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:21.940977 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c61080-8722-421d-9979-81c266da6d56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:21.942401 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c61080-8722-421d-9979-81c266da6d56 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:22.039372 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 602/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:26:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:26:22.045231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e62d8-bc86-4ff8-9121-0b1898d1bb96 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:22.141908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e62d8-bc86-4ff8-9121-0b1898d1bb96 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:22.175116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5a28b-51f3-4921-bf44-f4c5dd57409c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:22.175738 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5a28b-51f3-4921-bf44-f4c5dd57409c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:22.176024 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5a28b-51f3-4921-bf44-f4c5dd57409c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:22.176304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5a28b-51f3-4921-bf44-f4c5dd57409c None admin] PATH_INFO: `/v3/projects/16e919bbf5e74516a7d6ca748edcf419` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:22.177444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5a28b-51f3-4921-bf44-f4c5dd57409c None admin] RBAC: Authorizing `identity:get_project(project_id=16e919bbf5e74516a7d6ca748edcf419)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:22.178872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5a28b-51f3-4921-bf44-f4c5dd57409c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:22.180538 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 597/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Aug 1 07:26:22 2022] GET /identity/v3/projects/16e919bbf5e74516a7d6ca748edcf419 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:22.186494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c61080-8722-421d-9979-81c266da6d56 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:26:22.234441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:22.282315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c61080-8722-421d-9979-81c266da6d56 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:26:22.314123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd9e1c-af25-43d8-bf57-216e01e2087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:26:22.314719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd9e1c-af25-43d8-bf57-216e01e2087c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:26:22.315638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd9e1c-af25-43d8-bf57-216e01e2087c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:26:22.315912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd9e1c-af25-43d8-bf57-216e01e2087c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:26:22.316678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd9e1c-af25-43d8-bf57-216e01e2087c None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=16e919bbf5e74516a7d6ca748edcf419, user.id=c722c532efda462daf9bf110df255953) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:26:22.316984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd9e1c-af25-43d8-bf57-216e01e2087c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:26:22.318397 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd9e1c-af25-43d8-bf57-216e01e2087c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:26:22.324006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 603/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:26:22 2022] GET /identity/v3/role_assignments?user.id=c722c532efda462daf9bf110df255953&scope.project.id=16e919bbf5e74516a7d6ca748edcf419&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:26:32.171217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=71 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:26:32.321542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=43 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:17.289196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b4844-9ec3-4433-8306-98c348269023 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:17.289756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b4844-9ec3-4433-8306-98c348269023 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:17.289756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b4844-9ec3-4433-8306-98c348269023 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:17.291460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 598/1201] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:27:17.297823 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd36b52-3f21-45d0-ab7f-979da9f723c9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:17.298349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd36b52-3f21-45d0-ab7f-979da9f723c9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:17.298548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd36b52-3f21-45d0-ab7f-979da9f723c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:17.321279 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2dd36b52-3f21-45d0-ab7f-979da9f723c9 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:27:17.322503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2dd36b52-3f21-45d0-ab7f-979da9f723c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:27:17.410155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 604/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:27:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:27:17.438443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b4844-9ec3-4433-8306-98c348269023 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:17.575794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b4844-9ec3-4433-8306-98c348269023 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:17.623259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e29177-0122-4196-aee7-9d186e1fb329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:17.624257 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e29177-0122-4196-aee7-9d186e1fb329 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:17.624760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e29177-0122-4196-aee7-9d186e1fb329 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:17.625226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e29177-0122-4196-aee7-9d186e1fb329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:17.644819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e29177-0122-4196-aee7-9d186e1fb329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:17.647856 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e29177-0122-4196-aee7-9d186e1fb329 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-05e29177-0122-4196-aee7-9d186e1fb329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:17.670072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:17.676948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:17.678022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:17.678022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:17.678022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:17.678022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:17.678022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 599/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:17.682218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd36b52-3f21-45d0-ab7f-979da9f723c9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:17.832816 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd36b52-3f21-45d0-ab7f-979da9f723c9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:17.869953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:17.877593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf43a2-e44c-4fd2-adf5-511391ec5953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:17.878556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf43a2-e44c-4fd2-adf5-511391ec5953 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:17.878919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf43a2-e44c-4fd2-adf5-511391ec5953 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:17.879481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf43a2-e44c-4fd2-adf5-511391ec5953 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:17.880460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf43a2-e44c-4fd2-adf5-511391ec5953 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:17.881105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf43a2-e44c-4fd2-adf5-511391ec5953 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:17.883155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf43a2-e44c-4fd2-adf5-511391ec5953 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:17.900059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 605/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:17.906392 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e29177-0122-4196-aee7-9d186e1fb329 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:17.987669 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:18.049406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e29177-0122-4196-aee7-9d186e1fb329 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:18.092016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc710198-50a8-4b16-8963-053991f3b529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:18.092605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc710198-50a8-4b16-8963-053991f3b529 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:18.093039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc710198-50a8-4b16-8963-053991f3b529 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:18.093463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc710198-50a8-4b16-8963-053991f3b529 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:18.094151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc710198-50a8-4b16-8963-053991f3b529 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:18.095627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc710198-50a8-4b16-8963-053991f3b529 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:18.123463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 600/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Aug 1 07:27:17 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:27:20.463057 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf52bd6-04f4-4815-97f2-d42bc6737259 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:20.463628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf52bd6-04f4-4815-97f2-d42bc6737259 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:20.463628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf52bd6-04f4-4815-97f2-d42bc6737259 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:20.465118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 606/1206] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:27:20.471775 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b96892-be22-46f3-894d-5f6445316f22 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:20.473691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b96892-be22-46f3-894d-5f6445316f22 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:20.473691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b96892-be22-46f3-894d-5f6445316f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:20.492633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-49b96892-be22-46f3-894d-5f6445316f22 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:27:20.493830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-49b96892-be22-46f3-894d-5f6445316f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:27:20.570962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 601/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:27:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:27:20.598195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf52bd6-04f4-4815-97f2-d42bc6737259 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:20.714525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf52bd6-04f4-4815-97f2-d42bc6737259 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:20.762550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da46b9-47ce-4838-9bb0-8a0bea3915a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:20.763545 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da46b9-47ce-4838-9bb0-8a0bea3915a8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:20.763840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da46b9-47ce-4838-9bb0-8a0bea3915a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:20.764113 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da46b9-47ce-4838-9bb0-8a0bea3915a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:20.777073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da46b9-47ce-4838-9bb0-8a0bea3915a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-69da46b9-47ce-4838-9bb0-8a0bea3915a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:27:20.782081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:20.784334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:20.785205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 607/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:27:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:20.789811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b96892-be22-46f3-894d-5f6445316f22 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:20.834702 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:20.902536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b96892-be22-46f3-894d-5f6445316f22 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:20.940401 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc9f5c-dfc0-4455-b8c0-822a7773ae5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:20.941029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc9f5c-dfc0-4455-b8c0-822a7773ae5b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:20.941315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc9f5c-dfc0-4455-b8c0-822a7773ae5b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:20.941593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc9f5c-dfc0-4455-b8c0-822a7773ae5b None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:20.943249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc9f5c-dfc0-4455-b8c0-822a7773ae5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:20.943879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc9f5c-dfc0-4455-b8c0-822a7773ae5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:20.945750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc9f5c-dfc0-4455-b8c0-822a7773ae5b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:20.960615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 602/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:27:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:20.963292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da46b9-47ce-4838-9bb0-8a0bea3915a8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:21.024535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:21.078201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da46b9-47ce-4838-9bb0-8a0bea3915a8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:21.114524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2989ec-b487-42fe-bb15-086113d0de0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:21.115200 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2989ec-b487-42fe-bb15-086113d0de0e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:21.115485 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2989ec-b487-42fe-bb15-086113d0de0e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:21.115765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2989ec-b487-42fe-bb15-086113d0de0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:21.127336 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2989ec-b487-42fe-bb15-086113d0de0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:21.128787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2989ec-b487-42fe-bb15-086113d0de0e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ab2989ec-b487-42fe-bb15-086113d0de0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:21.144680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:21.145619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:21.146515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:21.146515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:21.146515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:21.146515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:21.148057 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 608/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:21.153939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc9f5c-dfc0-4455-b8c0-822a7773ae5b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:21.274035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:21.277554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc9f5c-dfc0-4455-b8c0-822a7773ae5b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:21.316621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd328c3c-cb43-4fb2-9621-065f390a6888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:21.317300 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd328c3c-cb43-4fb2-9621-065f390a6888 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:21.317549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd328c3c-cb43-4fb2-9621-065f390a6888 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:21.317811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd328c3c-cb43-4fb2-9621-065f390a6888 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:21.318496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd328c3c-cb43-4fb2-9621-065f390a6888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:21.318887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd328c3c-cb43-4fb2-9621-065f390a6888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:21.320243 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd328c3c-cb43-4fb2-9621-065f390a6888 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:21.333313 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 603/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:27:21.339233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2989ec-b487-42fe-bb15-086113d0de0e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:21.462966 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2989ec-b487-42fe-bb15-086113d0de0e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:21.464673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:21.506950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162589a-a838-43e0-bb94-e8576c6a33da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:21.507626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162589a-a838-43e0-bb94-e8576c6a33da None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:21.507912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162589a-a838-43e0-bb94-e8576c6a33da None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:21.508264 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162589a-a838-43e0-bb94-e8576c6a33da None admin] PATH_INFO: `/v3/users/placement` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:21.516991 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162589a-a838-43e0-bb94-e8576c6a33da None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:21.518595 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162589a-a838-43e0-bb94-e8576c6a33da None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-1162589a-a838-43e0-bb94-e8576c6a33da None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:27:21.531605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:21.532631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:21.534417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:21.534417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:21.534417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:27:21.534417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:27:21.534417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:27:21.534417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:27:21.534417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:27:21.534417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:27:21.534417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:27:21.534417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:21.534417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 609/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:21.539232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd328c3c-cb43-4fb2-9621-065f390a6888 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:21.662649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd328c3c-cb43-4fb2-9621-065f390a6888 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:21.711704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf54a6-a9b7-4ac7-b212-b240fd286103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:21.712473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf54a6-a9b7-4ac7-b212-b240fd286103 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:21.712759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf54a6-a9b7-4ac7-b212-b240fd286103 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:21.713032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf54a6-a9b7-4ac7-b212-b240fd286103 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:21.713860 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf54a6-a9b7-4ac7-b212-b240fd286103 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:21.714171 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf54a6-a9b7-4ac7-b212-b240fd286103 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:21.715691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf54a6-a9b7-4ac7-b212-b240fd286103 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:21.736900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 604/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Aug 1 07:27:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:21.743174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162589a-a838-43e0-bb94-e8576c6a33da None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:21.870268 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162589a-a838-43e0-bb94-e8576c6a33da None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:21.912056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ddb6c-87f1-4cfe-8c3d-c110e5745db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:21.913091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ddb6c-87f1-4cfe-8c3d-c110e5745db9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:21.913599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ddb6c-87f1-4cfe-8c3d-c110e5745db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:21.913942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ddb6c-87f1-4cfe-8c3d-c110e5745db9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:21.950473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:21.994318 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ddb6c-87f1-4cfe-8c3d-c110e5745db9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:21.995881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ddb6c-87f1-4cfe-8c3d-c110e5745db9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:22.138119 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 610/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:27:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:27:22.140079 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf54a6-a9b7-4ac7-b212-b240fd286103 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:22.165317 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:22.269654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf54a6-a9b7-4ac7-b212-b240fd286103 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:22.309463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bff13-0d9b-4bab-bed9-14110ab8da02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:22.309839 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bff13-0d9b-4bab-bed9-14110ab8da02 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:22.310521 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bff13-0d9b-4bab-bed9-14110ab8da02 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:22.310521 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bff13-0d9b-4bab-bed9-14110ab8da02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:22.323837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bff13-0d9b-4bab-bed9-14110ab8da02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:22.325522 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bff13-0d9b-4bab-bed9-14110ab8da02 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-114bff13-0d9b-4bab-bed9-14110ab8da02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:22.348609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:22.349629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:22.353067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:22.353067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:22.353067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:22.353067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:22.353067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 605/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:27:22.356139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ddb6c-87f1-4cfe-8c3d-c110e5745db9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:22.434446 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:22.476888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ddb6c-87f1-4cfe-8c3d-c110e5745db9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:22.516787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0559ae2-379c-482b-b192-243c762e37a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:22.517415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0559ae2-379c-482b-b192-243c762e37a8 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:22.517703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0559ae2-379c-482b-b192-243c762e37a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:22.518002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0559ae2-379c-482b-b192-243c762e37a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:22.518674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0559ae2-379c-482b-b192-243c762e37a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:22.518995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0559ae2-379c-482b-b192-243c762e37a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:22.520503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0559ae2-379c-482b-b192-243c762e37a8 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:22.538270 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 611/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:22.539512 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bff13-0d9b-4bab-bed9-14110ab8da02 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:22.615496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:22.659297 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bff13-0d9b-4bab-bed9-14110ab8da02 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:22.698751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5f48a-77bf-4024-81e4-40a00cfdf130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:22.699365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5f48a-77bf-4024-81e4-40a00cfdf130 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:22.699653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5f48a-77bf-4024-81e4-40a00cfdf130 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:22.699932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5f48a-77bf-4024-81e4-40a00cfdf130 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:22.707854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5f48a-77bf-4024-81e4-40a00cfdf130 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:22.709445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5f48a-77bf-4024-81e4-40a00cfdf130 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ddb5f48a-77bf-4024-81e4-40a00cfdf130 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:27:22.722108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:22.727045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:22.729124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:27:22.729124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:27:22.729124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:27:22.729124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:27:22.729124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:27:22.729124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:27:22.729124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:27:22.729124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:22.729124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 606/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:27:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:22.736796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0559ae2-379c-482b-b192-243c762e37a8 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:22.859752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0559ae2-379c-482b-b192-243c762e37a8 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:22.883447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:22.898120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3f95f-17e4-4a5f-93b1-54acc69748cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:22.898743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3f95f-17e4-4a5f-93b1-54acc69748cd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:22.899034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3f95f-17e4-4a5f-93b1-54acc69748cd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:22.899363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3f95f-17e4-4a5f-93b1-54acc69748cd None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:22.900093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3f95f-17e4-4a5f-93b1-54acc69748cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:22.900421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3f95f-17e4-4a5f-93b1-54acc69748cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:22.901868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3f95f-17e4-4a5f-93b1-54acc69748cd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:22.920437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 612/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:27:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:22.922893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5f48a-77bf-4024-81e4-40a00cfdf130 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:23.046057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5f48a-77bf-4024-81e4-40a00cfdf130 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:23.081617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:23.094196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c109f-a123-45ac-8f9d-4e01fbeab56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:23.094823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c109f-a123-45ac-8f9d-4e01fbeab56f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:23.095145 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c109f-a123-45ac-8f9d-4e01fbeab56f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:23.095434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c109f-a123-45ac-8f9d-4e01fbeab56f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:23.096374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c109f-a123-45ac-8f9d-4e01fbeab56f None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=928c5d1284b74f1291c031323bb03a20) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:23.096713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c109f-a123-45ac-8f9d-4e01fbeab56f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:23.098171 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c109f-a123-45ac-8f9d-4e01fbeab56f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:23.111784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 607/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:27:22 2022] GET /identity/v3/role_assignments?user.id=928c5d1284b74f1291c031323bb03a20&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:25.134986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5043a-544b-44ba-85b3-80d2b45e8ff3 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:25.135592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5043a-544b-44ba-85b3-80d2b45e8ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:25.135592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5043a-544b-44ba-85b3-80d2b45e8ff3 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:25.136932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 613/1220] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:27:25.142924 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea866512-ddfe-4bf1-9a98-add2c3b6e9b3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:25.143395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea866512-ddfe-4bf1-9a98-add2c3b6e9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:25.143524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea866512-ddfe-4bf1-9a98-add2c3b6e9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:25.162307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ea866512-ddfe-4bf1-9a98-add2c3b6e9b3 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:27:25.163522 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ea866512-ddfe-4bf1-9a98-add2c3b6e9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:27:25.241359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 608/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:27:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 07:27:25.263726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5043a-544b-44ba-85b3-80d2b45e8ff3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:25.367737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5043a-544b-44ba-85b3-80d2b45e8ff3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:25.405548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c5b01-ba09-4e52-ab0d-5d8580bcaf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:25.406230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c5b01-ba09-4e52-ab0d-5d8580bcaf04 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:25.406506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c5b01-ba09-4e52-ab0d-5d8580bcaf04 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:25.406778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c5b01-ba09-4e52-ab0d-5d8580bcaf04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:25.416745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c5b01-ba09-4e52-ab0d-5d8580bcaf04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-1e6c5b01-ba09-4e52-ab0d-5d8580bcaf04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:27:25.421656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:25.422784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:25.423702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 614/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:27:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:25.429468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea866512-ddfe-4bf1-9a98-add2c3b6e9b3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:25.534480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea866512-ddfe-4bf1-9a98-add2c3b6e9b3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:25.560253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:25.572629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3f2e0-3760-4cc6-922e-948cafaf8e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:25.573250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3f2e0-3760-4cc6-922e-948cafaf8e51 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:25.573644 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3f2e0-3760-4cc6-922e-948cafaf8e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:25.573888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3f2e0-3760-4cc6-922e-948cafaf8e51 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:25.574653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3f2e0-3760-4cc6-922e-948cafaf8e51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:25.574939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3f2e0-3760-4cc6-922e-948cafaf8e51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:25.576460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3f2e0-3760-4cc6-922e-948cafaf8e51 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:25.584985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 609/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:27:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:25.590802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c5b01-ba09-4e52-ab0d-5d8580bcaf04 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:25.638120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:25.701893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c5b01-ba09-4e52-ab0d-5d8580bcaf04 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:25.736980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df38b2-4036-4ccf-b3bd-5ba201cd1755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:25.737690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df38b2-4036-4ccf-b3bd-5ba201cd1755 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:25.737998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df38b2-4036-4ccf-b3bd-5ba201cd1755 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:25.738277 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df38b2-4036-4ccf-b3bd-5ba201cd1755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:25.749102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df38b2-4036-4ccf-b3bd-5ba201cd1755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:25.750627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df38b2-4036-4ccf-b3bd-5ba201cd1755 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-e4df38b2-4036-4ccf-b3bd-5ba201cd1755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:25.766565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:25.767806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:25.768693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:25.768693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:25.768693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:25.768693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:25.768693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 615/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:25.773624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3f2e0-3760-4cc6-922e-948cafaf8e51 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:25.888303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3f2e0-3760-4cc6-922e-948cafaf8e51 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:25.926133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25897c-c593-41f1-8c87-e9aa1e091ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:25.926747 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25897c-c593-41f1-8c87-e9aa1e091ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:25.927029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25897c-c593-41f1-8c87-e9aa1e091ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:25.927335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25897c-c593-41f1-8c87-e9aa1e091ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:25.928032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25897c-c593-41f1-8c87-e9aa1e091ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:25.928336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25897c-c593-41f1-8c87-e9aa1e091ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:25.929706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25897c-c593-41f1-8c87-e9aa1e091ee2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:25.941950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 610/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:25.948963 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df38b2-4036-4ccf-b3bd-5ba201cd1755 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:26.049237 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:26.056594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df38b2-4036-4ccf-b3bd-5ba201cd1755 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:26.091124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d316d-725c-4a3b-a4d8-769f3a6db9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:26.091761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d316d-725c-4a3b-a4d8-769f3a6db9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:26.092140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d316d-725c-4a3b-a4d8-769f3a6db9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:26.092521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d316d-725c-4a3b-a4d8-769f3a6db9c6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:26.100361 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d316d-725c-4a3b-a4d8-769f3a6db9c6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:26.101793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d316d-725c-4a3b-a4d8-769f3a6db9c6 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-a66d316d-725c-4a3b-a4d8-769f3a6db9c6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:27:26.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:26.114572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:26.115619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:26.115619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:26.115619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:27:26.115619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:27:26.115619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:27:26.115619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:27:26.115619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:27:26.115619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:27:26.115619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:27:26.115619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:26.115619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 616/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:26.120483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25897c-c593-41f1-8c87-e9aa1e091ee2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:26.168705 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:26.227256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25897c-c593-41f1-8c87-e9aa1e091ee2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:26.264576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d68e25-ed8c-4d01-a957-53ae9c9cb1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:26.265592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d68e25-ed8c-4d01-a957-53ae9c9cb1fb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:26.265878 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d68e25-ed8c-4d01-a957-53ae9c9cb1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:26.266154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d68e25-ed8c-4d01-a957-53ae9c9cb1fb None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:26.266947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d68e25-ed8c-4d01-a957-53ae9c9cb1fb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:26.267301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d68e25-ed8c-4d01-a957-53ae9c9cb1fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:26.268731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d68e25-ed8c-4d01-a957-53ae9c9cb1fb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:26.286337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 611/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Aug 1 07:27:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:26.292051 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d316d-725c-4a3b-a4d8-769f3a6db9c6 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:26.401164 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d316d-725c-4a3b-a4d8-769f3a6db9c6 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:26.445703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267481b4-07fa-49a1-8da8-3801b7019a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:26.446384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267481b4-07fa-49a1-8da8-3801b7019a0f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:26.446772 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267481b4-07fa-49a1-8da8-3801b7019a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:26.447122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267481b4-07fa-49a1-8da8-3801b7019a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:26.461376 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267481b4-07fa-49a1-8da8-3801b7019a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:26.462904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267481b4-07fa-49a1-8da8-3801b7019a0f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-267481b4-07fa-49a1-8da8-3801b7019a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:26.478068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:26.479159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:26.480086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:26.480086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:26.480086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:26.480086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:26.484533 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 617/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:26.485587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d68e25-ed8c-4d01-a957-53ae9c9cb1fb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:26.596548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d68e25-ed8c-4d01-a957-53ae9c9cb1fb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:26.599651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:26.636854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768f577-91c3-44c5-8662-816fc8029ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:26.637591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768f577-91c3-44c5-8662-816fc8029ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:26.638001 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768f577-91c3-44c5-8662-816fc8029ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:26.638261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768f577-91c3-44c5-8662-816fc8029ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:26.639142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768f577-91c3-44c5-8662-816fc8029ca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:26.639432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768f577-91c3-44c5-8662-816fc8029ca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:26.641178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768f577-91c3-44c5-8662-816fc8029ca3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:26.655938 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 612/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:26.661642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267481b4-07fa-49a1-8da8-3801b7019a0f None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:26.674059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:26.777906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267481b4-07fa-49a1-8da8-3801b7019a0f None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:26.816994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ebf99-5388-4aaf-9992-236932488e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:26.817664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ebf99-5388-4aaf-9992-236932488e2c None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:26.817918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ebf99-5388-4aaf-9992-236932488e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:26.818197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ebf99-5388-4aaf-9992-236932488e2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:26.826593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ebf99-5388-4aaf-9992-236932488e2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:26.828071 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ebf99-5388-4aaf-9992-236932488e2c None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ed9ebf99-5388-4aaf-9992-236932488e2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:27:26.839552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:26.841822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:26.844248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:27:26.844248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:27:26.844248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:27:26.844248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:27:26.844248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:27:26.844248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:27:26.844248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:27:26.844248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:26.844248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 618/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:27:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:26.847836 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768f577-91c3-44c5-8662-816fc8029ca3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:26.958434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768f577-91c3-44c5-8662-816fc8029ca3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:26.999936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61ce45-5d66-407e-b1c3-a8873a85325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:27.000556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61ce45-5d66-407e-b1c3-a8873a85325c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:27.000839 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61ce45-5d66-407e-b1c3-a8873a85325c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:27.001115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61ce45-5d66-407e-b1c3-a8873a85325c None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:27.001864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61ce45-5d66-407e-b1c3-a8873a85325c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:27.002179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61ce45-5d66-407e-b1c3-a8873a85325c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:27.003683 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61ce45-5d66-407e-b1c3-a8873a85325c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:27.019099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 613/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:27:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:27.025164 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ebf99-5388-4aaf-9992-236932488e2c None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:27.097636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:27.142985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ebf99-5388-4aaf-9992-236932488e2c None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:27.181986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef53b9-cc25-455e-87c5-a9f735a2a037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:27.182657 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef53b9-cc25-455e-87c5-a9f735a2a037 None admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:27.182943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef53b9-cc25-455e-87c5-a9f735a2a037 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:27.183264 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef53b9-cc25-455e-87c5-a9f735a2a037 None admin] PATH_INFO: `/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/928c5d1284b74f1291c031323bb03a20/roles/74131e9234dc48a2b55332d15e8c6462` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:27.198040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef53b9-cc25-455e-87c5-a9f735a2a037 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ddf7d6f2424a85a741d017497b49c9, user_id=928c5d1284b74f1291c031323bb03a20, role_id=74131e9234dc48a2b55332d15e8c6462)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:27.199525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef53b9-cc25-455e-87c5-a9f735a2a037 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:27.212933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 619/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:27:27 2022] PUT /identity/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/928c5d1284b74f1291c031323bb03a20/roles/74131e9234dc48a2b55332d15e8c6462 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:27:29.309022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba93acb-26b7-49b9-ad64-7d93f72c3a15 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:29.309551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba93acb-26b7-49b9-ad64-7d93f72c3a15 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:29.309551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba93acb-26b7-49b9-ad64-7d93f72c3a15 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:29.310939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 614/1233] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:27:29.316429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3581be1-9249-485a-b25b-010fb8738c20 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:29.316840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3581be1-9249-485a-b25b-010fb8738c20 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:29.316986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3581be1-9249-485a-b25b-010fb8738c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:29.336781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f3581be1-9249-485a-b25b-010fb8738c20 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:27:29.338133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f3581be1-9249-485a-b25b-010fb8738c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:27:29.414570 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 620/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:27:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:27:29.437675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba93acb-26b7-49b9-ad64-7d93f72c3a15 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:29.465409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:29.548610 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba93acb-26b7-49b9-ad64-7d93f72c3a15 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:29.586335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a41799-c7c9-4723-bfef-ddcfe1a69fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:29.586903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a41799-c7c9-4723-bfef-ddcfe1a69fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:29.587295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a41799-c7c9-4723-bfef-ddcfe1a69fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:29.587515 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a41799-c7c9-4723-bfef-ddcfe1a69fd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:29.598617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a41799-c7c9-4723-bfef-ddcfe1a69fd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-54a41799-c7c9-4723-bfef-ddcfe1a69fd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:27:29.603755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:29.604983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:29.606411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 615/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Aug 1 07:27:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:29.611194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3581be1-9249-485a-b25b-010fb8738c20 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:29.666007 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:29.726643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3581be1-9249-485a-b25b-010fb8738c20 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:29.763549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62634610-b25e-4663-8927-4570e3e6784e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:29.764876 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62634610-b25e-4663-8927-4570e3e6784e None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:29.765147 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62634610-b25e-4663-8927-4570e3e6784e None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:29.765505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62634610-b25e-4663-8927-4570e3e6784e None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:29.766283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62634610-b25e-4663-8927-4570e3e6784e None admin] RBAC: Adding query filter params (name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:29.766593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62634610-b25e-4663-8927-4570e3e6784e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:29.768123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62634610-b25e-4663-8927-4570e3e6784e None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:29.777488 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 621/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Aug 1 07:27:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:29.783515 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a41799-c7c9-4723-bfef-ddcfe1a69fd8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:29.880577 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:29.899659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a41799-c7c9-4723-bfef-ddcfe1a69fd8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:29.938416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50caad-a60e-4d26-8349-a756684a8a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:29.939089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50caad-a60e-4d26-8349-a756684a8a54 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:29.939375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50caad-a60e-4d26-8349-a756684a8a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:29.939659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50caad-a60e-4d26-8349-a756684a8a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:29.952085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50caad-a60e-4d26-8349-a756684a8a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:29.953652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50caad-a60e-4d26-8349-a756684a8a54 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2c50caad-a60e-4d26-8349-a756684a8a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:29.971447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:29.972720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:29.978687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:29.978687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:29.978687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:29.978687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:29.978687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 616/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:29.981664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62634610-b25e-4663-8927-4570e3e6784e None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:30.094319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62634610-b25e-4663-8927-4570e3e6784e None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:30.103985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:30.134011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9a919-0a17-4fb8-aad8-1e4d20f1fd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:30.135008 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9a919-0a17-4fb8-aad8-1e4d20f1fd9b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:30.135631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9a919-0a17-4fb8-aad8-1e4d20f1fd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:30.136178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9a919-0a17-4fb8-aad8-1e4d20f1fd9b None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:30.137460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9a919-0a17-4fb8-aad8-1e4d20f1fd9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:30.139499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9a919-0a17-4fb8-aad8-1e4d20f1fd9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:30.140798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9a919-0a17-4fb8-aad8-1e4d20f1fd9b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:30.155692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 622/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:30.161571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50caad-a60e-4d26-8349-a756684a8a54 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:30.269072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50caad-a60e-4d26-8349-a756684a8a54 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:30.308462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373de897-f1e9-4b16-a148-10082ddc2863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:30.309243 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373de897-f1e9-4b16-a148-10082ddc2863 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:30.309781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373de897-f1e9-4b16-a148-10082ddc2863 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:30.309865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373de897-f1e9-4b16-a148-10082ddc2863 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:30.313497 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:30.318903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373de897-f1e9-4b16-a148-10082ddc2863 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:30.321109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373de897-f1e9-4b16-a148-10082ddc2863 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-373de897-f1e9-4b16-a148-10082ddc2863 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:27:30.333550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:30.334732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:30.335798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:30.335798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:30.335798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:27:30.335798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:27:30.335798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:27:30.335798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:27:30.335798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:27:30.335798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:27:30.335798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:27:30.335798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:30.335798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 617/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:30.341226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9a919-0a17-4fb8-aad8-1e4d20f1fd9b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:30.447550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9a919-0a17-4fb8-aad8-1e4d20f1fd9b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:30.484195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152de3a-4e97-4b31-8e0f-b701da0170dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:30.484821 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152de3a-4e97-4b31-8e0f-b701da0170dc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:30.485107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152de3a-4e97-4b31-8e0f-b701da0170dc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:30.485387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152de3a-4e97-4b31-8e0f-b701da0170dc None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:30.486184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152de3a-4e97-4b31-8e0f-b701da0170dc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:30.486498 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152de3a-4e97-4b31-8e0f-b701da0170dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:30.487968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152de3a-4e97-4b31-8e0f-b701da0170dc None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:30.511145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 623/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Aug 1 07:27:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:30.512719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373de897-f1e9-4b16-a148-10082ddc2863 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:30.623317 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373de897-f1e9-4b16-a148-10082ddc2863 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:30.663233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9075b6-5932-4abc-818a-f2b7f5a95002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:30.663944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9075b6-5932-4abc-818a-f2b7f5a95002 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:30.664291 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9075b6-5932-4abc-818a-f2b7f5a95002 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:30.664675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9075b6-5932-4abc-818a-f2b7f5a95002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:30.735818 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:30.743525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9075b6-5932-4abc-818a-f2b7f5a95002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:30.745268 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9075b6-5932-4abc-818a-f2b7f5a95002 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:30.863597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 618/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:27:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:27:30.869024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152de3a-4e97-4b31-8e0f-b701da0170dc None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:30.900499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:30.987111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152de3a-4e97-4b31-8e0f-b701da0170dc None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:31.023449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f20efb-49d1-47fa-94b4-f536b9cef491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:31.024095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f20efb-49d1-47fa-94b4-f536b9cef491 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:31.024381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f20efb-49d1-47fa-94b4-f536b9cef491 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:31.024688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f20efb-49d1-47fa-94b4-f536b9cef491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:31.035793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f20efb-49d1-47fa-94b4-f536b9cef491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:31.037503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f20efb-49d1-47fa-94b4-f536b9cef491 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-58f20efb-49d1-47fa-94b4-f536b9cef491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:31.054708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:31.055927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:31.057035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:31.057035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:31.057035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:31.057035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:31.057035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 624/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:31.062334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9075b6-5932-4abc-818a-f2b7f5a95002 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:31.161383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:31.169331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9075b6-5932-4abc-818a-f2b7f5a95002 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:31.207094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfdcd9-31bf-4923-9e7d-3643c394e90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:31.207773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfdcd9-31bf-4923-9e7d-3643c394e90c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:31.208089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfdcd9-31bf-4923-9e7d-3643c394e90c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:31.208367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfdcd9-31bf-4923-9e7d-3643c394e90c None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:31.209047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfdcd9-31bf-4923-9e7d-3643c394e90c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:31.209353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfdcd9-31bf-4923-9e7d-3643c394e90c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:31.210706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfdcd9-31bf-4923-9e7d-3643c394e90c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:31.223058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 619/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:27:31.228837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f20efb-49d1-47fa-94b4-f536b9cef491 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:31.314813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:31.337977 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f20efb-49d1-47fa-94b4-f536b9cef491 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:31.375717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f599226-504c-41e6-b986-8ca10202e807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:31.376429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f599226-504c-41e6-b986-8ca10202e807 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:31.376679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f599226-504c-41e6-b986-8ca10202e807 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:31.377023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f599226-504c-41e6-b986-8ca10202e807 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:31.384259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f599226-504c-41e6-b986-8ca10202e807 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:31.385877 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f599226-504c-41e6-b986-8ca10202e807 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-3f599226-504c-41e6-b986-8ca10202e807 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:27:31.398893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:31.399995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:31.400902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:27:31.400902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:27:31.400902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:27:31.400902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:27:31.400902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:27:31.400902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:27:31.400902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:27:31.400902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:31.400902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 625/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:27:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:31.406932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfdcd9-31bf-4923-9e7d-3643c394e90c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:31.517630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfdcd9-31bf-4923-9e7d-3643c394e90c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:31.557885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1499a2-a56d-4f19-a297-2302d4be3c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:31.558513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1499a2-a56d-4f19-a297-2302d4be3c21 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:31.558814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1499a2-a56d-4f19-a297-2302d4be3c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:31.559097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1499a2-a56d-4f19-a297-2302d4be3c21 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:31.559845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1499a2-a56d-4f19-a297-2302d4be3c21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:31.560155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1499a2-a56d-4f19-a297-2302d4be3c21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:31.562304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1499a2-a56d-4f19-a297-2302d4be3c21 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:31.577317 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 620/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:27:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:27:31.583438 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f599226-504c-41e6-b986-8ca10202e807 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:31.703536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f599226-504c-41e6-b986-8ca10202e807 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:31.741307 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0119f5-0d96-4ab9-aa2d-44b09a45c127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:31.741928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0119f5-0d96-4ab9-aa2d-44b09a45c127 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:31.742213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0119f5-0d96-4ab9-aa2d-44b09a45c127 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:31.742511 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0119f5-0d96-4ab9-aa2d-44b09a45c127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:31.743442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0119f5-0d96-4ab9-aa2d-44b09a45c127 None admin] RBAC: Adding query filter params (role.id=74131e9234dc48a2b55332d15e8c6462, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=928c5d1284b74f1291c031323bb03a20) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:31.743761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0119f5-0d96-4ab9-aa2d-44b09a45c127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:31.745187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0119f5-0d96-4ab9-aa2d-44b09a45c127 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:31.751630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 626/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:27:31 2022] GET /identity/v3/role_assignments?user.id=928c5d1284b74f1291c031323bb03a20&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=74131e9234dc48a2b55332d15e8c6462 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:34.031359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23178dfa-5931-4201-bd0e-774dd553f1da None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:34.031925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23178dfa-5931-4201-bd0e-774dd553f1da None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:34.031925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23178dfa-5931-4201-bd0e-774dd553f1da None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:34.033445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 621/1247] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:27:34.039611 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1994b4-12b8-4a1c-a500-2d3322ad571a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:34.040038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1994b4-12b8-4a1c-a500-2d3322ad571a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:34.040274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1994b4-12b8-4a1c-a500-2d3322ad571a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:34.048174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:34.063011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6e1994b4-12b8-4a1c-a500-2d3322ad571a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:27:34.064273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6e1994b4-12b8-4a1c-a500-2d3322ad571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:27:34.140137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 627/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:27:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:27:34.168393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23178dfa-5931-4201-bd0e-774dd553f1da None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:34.188515 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:34.285776 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23178dfa-5931-4201-bd0e-774dd553f1da None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:34.321261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5652b-5276-4a16-8247-be172256a6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:34.321907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5652b-5276-4a16-8247-be172256a6ff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:34.322196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5652b-5276-4a16-8247-be172256a6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:34.322468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5652b-5276-4a16-8247-be172256a6ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:34.333940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5652b-5276-4a16-8247-be172256a6ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b8e5652b-5276-4a16-8247-be172256a6ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:27:34.338814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:34.340163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:34.341081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 622/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:27:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:34.347420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1994b4-12b8-4a1c-a500-2d3322ad571a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:34.463823 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1994b4-12b8-4a1c-a500-2d3322ad571a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:34.498514 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:34.504716 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d375a0-5c40-4344-84b2-93004bd04b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:34.505851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d375a0-5c40-4344-84b2-93004bd04b1b None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:34.506186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d375a0-5c40-4344-84b2-93004bd04b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:34.506476 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d375a0-5c40-4344-84b2-93004bd04b1b None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:34.507409 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d375a0-5c40-4344-84b2-93004bd04b1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:34.507787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d375a0-5c40-4344-84b2-93004bd04b1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:34.509617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d375a0-5c40-4344-84b2-93004bd04b1b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:34.519371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 628/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:27:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:34.525050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5652b-5276-4a16-8247-be172256a6ff None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:34.615209 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:34.641902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5652b-5276-4a16-8247-be172256a6ff None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:34.686309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a342420-50cb-4cb3-a61a-b54eb1870bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:34.687134 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a342420-50cb-4cb3-a61a-b54eb1870bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:34.687461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a342420-50cb-4cb3-a61a-b54eb1870bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:34.687816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a342420-50cb-4cb3-a61a-b54eb1870bd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:34.699524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a342420-50cb-4cb3-a61a-b54eb1870bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:34.701274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a342420-50cb-4cb3-a61a-b54eb1870bd2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4a342420-50cb-4cb3-a61a-b54eb1870bd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:34.717509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:34.718648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:34.719632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:34.719632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:34.719632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:34.719632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:34.719632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 623/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:34.725373 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d375a0-5c40-4344-84b2-93004bd04b1b None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:34.842857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d375a0-5c40-4344-84b2-93004bd04b1b None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:34.880445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8de789-d4a5-4ec0-a24a-dde5366ea4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:34.880953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8de789-d4a5-4ec0-a24a-dde5366ea4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:34.881227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8de789-d4a5-4ec0-a24a-dde5366ea4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:34.881510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8de789-d4a5-4ec0-a24a-dde5366ea4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:34.882263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8de789-d4a5-4ec0-a24a-dde5366ea4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:34.882520 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8de789-d4a5-4ec0-a24a-dde5366ea4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:34.883930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8de789-d4a5-4ec0-a24a-dde5366ea4f2 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:34.896766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 629/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:34.902515 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a342420-50cb-4cb3-a61a-b54eb1870bd2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:35.018377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a342420-50cb-4cb3-a61a-b54eb1870bd2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:35.047722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:35.057269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75889dd1-1376-45f9-8884-3514ac017821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:35.057920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75889dd1-1376-45f9-8884-3514ac017821 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:35.058302 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75889dd1-1376-45f9-8884-3514ac017821 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:35.058568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75889dd1-1376-45f9-8884-3514ac017821 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:35.068076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75889dd1-1376-45f9-8884-3514ac017821 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:35.069515 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75889dd1-1376-45f9-8884-3514ac017821 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-75889dd1-1376-45f9-8884-3514ac017821 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:27:35.082673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:35.083830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:35.084945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:35.084945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:35.084945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:27:35.084945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:27:35.084945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:27:35.084945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:27:35.084945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:27:35.084945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:27:35.084945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:27:35.084945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:35.084945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 624/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:35.089728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8de789-d4a5-4ec0-a24a-dde5366ea4f2 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:35.151831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:35.201625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8de789-d4a5-4ec0-a24a-dde5366ea4f2 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:35.238617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d91a03-c7d0-4620-b974-3fe0b59c0db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:35.239294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d91a03-c7d0-4620-b974-3fe0b59c0db0 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:35.239565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d91a03-c7d0-4620-b974-3fe0b59c0db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:35.239849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d91a03-c7d0-4620-b974-3fe0b59c0db0 None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:35.240646 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d91a03-c7d0-4620-b974-3fe0b59c0db0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:35.240960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d91a03-c7d0-4620-b974-3fe0b59c0db0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:35.242397 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d91a03-c7d0-4620-b974-3fe0b59c0db0 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:35.261543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 630/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Aug 1 07:27:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:27:35.268140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75889dd1-1376-45f9-8884-3514ac017821 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:35.381488 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75889dd1-1376-45f9-8884-3514ac017821 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:35.418716 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabff08-cd5f-475e-acc0-7b0f968f5491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:35.419378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabff08-cd5f-475e-acc0-7b0f968f5491 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:35.419809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabff08-cd5f-475e-acc0-7b0f968f5491 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:35.420009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabff08-cd5f-475e-acc0-7b0f968f5491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:35.477860 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:35.491748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabff08-cd5f-475e-acc0-7b0f968f5491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:35.493221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabff08-cd5f-475e-acc0-7b0f968f5491 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:35.602311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 625/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:27:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:27:35.608894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d91a03-c7d0-4620-b974-3fe0b59c0db0 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:35.725503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d91a03-c7d0-4620-b974-3fe0b59c0db0 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:35.741084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:35.765205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6ef03-ba8b-4b7e-b72c-e2c30fb240a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:35.766101 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6ef03-ba8b-4b7e-b72c-e2c30fb240a7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:35.766385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6ef03-ba8b-4b7e-b72c-e2c30fb240a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:35.766660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6ef03-ba8b-4b7e-b72c-e2c30fb240a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:35.777380 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6ef03-ba8b-4b7e-b72c-e2c30fb240a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:35.778850 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6ef03-ba8b-4b7e-b72c-e2c30fb240a7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-0ed6ef03-ba8b-4b7e-b72c-e2c30fb240a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:35.795429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:35.796427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:35.798387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:35.798387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:35.798387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:35.798387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:35.798387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 631/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 07:27:35.802696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabff08-cd5f-475e-acc0-7b0f968f5491 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:35.900785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:35.916348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabff08-cd5f-475e-acc0-7b0f968f5491 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:35.956762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862a96a-f7b5-47ab-a8fa-f6d13dd59d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:35.957429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862a96a-f7b5-47ab-a8fa-f6d13dd59d6b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:35.957741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862a96a-f7b5-47ab-a8fa-f6d13dd59d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:35.958033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862a96a-f7b5-47ab-a8fa-f6d13dd59d6b None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:35.958759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862a96a-f7b5-47ab-a8fa-f6d13dd59d6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:35.959089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862a96a-f7b5-47ab-a8fa-f6d13dd59d6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:35.960498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862a96a-f7b5-47ab-a8fa-f6d13dd59d6b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:35.973584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 626/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:35.979772 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6ef03-ba8b-4b7e-b72c-e2c30fb240a7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:36.090169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6ef03-ba8b-4b7e-b72c-e2c30fb240a7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:36.128353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713faa2-8093-4483-b72d-8985a37a8198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:36.129123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713faa2-8093-4483-b72d-8985a37a8198 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:36.129421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713faa2-8093-4483-b72d-8985a37a8198 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:36.129712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713faa2-8093-4483-b72d-8985a37a8198 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:36.138087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713faa2-8093-4483-b72d-8985a37a8198 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:36.140701 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713faa2-8093-4483-b72d-8985a37a8198 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9713faa2-8093-4483-b72d-8985a37a8198 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:27:36.153186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:36.154818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:36.156030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:27:36.156030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:27:36.156030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:27:36.156030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:27:36.156030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:27:36.156030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:27:36.156030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:27:36.156030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:36.156030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 632/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:27:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:36.161316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862a96a-f7b5-47ab-a8fa-f6d13dd59d6b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:36.264960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862a96a-f7b5-47ab-a8fa-f6d13dd59d6b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:36.302055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ac421-db05-46ae-bbb2-521a16307f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:36.302710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ac421-db05-46ae-bbb2-521a16307f95 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:36.303001 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ac421-db05-46ae-bbb2-521a16307f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:36.303314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ac421-db05-46ae-bbb2-521a16307f95 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:36.304186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ac421-db05-46ae-bbb2-521a16307f95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:36.304490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ac421-db05-46ae-bbb2-521a16307f95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:36.306165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ac421-db05-46ae-bbb2-521a16307f95 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:36.319907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 627/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:27:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:36.327045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713faa2-8093-4483-b72d-8985a37a8198 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:36.355851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:36.438309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713faa2-8093-4483-b72d-8985a37a8198 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:36.476126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1b964-d746-4e26-a7a9-8a955ac8f49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:36.476755 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1b964-d746-4e26-a7a9-8a955ac8f49f None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:36.477047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1b964-d746-4e26-a7a9-8a955ac8f49f None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:36.477318 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1b964-d746-4e26-a7a9-8a955ac8f49f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:36.478105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1b964-d746-4e26-a7a9-8a955ac8f49f None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=928c5d1284b74f1291c031323bb03a20) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:36.478437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1b964-d746-4e26-a7a9-8a955ac8f49f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:36.479887 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1b964-d746-4e26-a7a9-8a955ac8f49f None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:36.486083 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 633/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:27:36 2022] GET /identity/v3/role_assignments?user.id=928c5d1284b74f1291c031323bb03a20&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:38.523290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f222d-efde-48ff-a1a8-2bcb9a46ed1d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:38.523814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f222d-efde-48ff-a1a8-2bcb9a46ed1d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:38.523936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f222d-efde-48ff-a1a8-2bcb9a46ed1d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:38.525200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 628/1261] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:27:38.530406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a86f2-6c2e-46b1-87be-c43c9ba2c24f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:38.530825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a86f2-6c2e-46b1-87be-c43c9ba2c24f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:38.530997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a86f2-6c2e-46b1-87be-c43c9ba2c24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:38.550042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f85a86f2-6c2e-46b1-87be-c43c9ba2c24f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:27:38.551250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f85a86f2-6c2e-46b1-87be-c43c9ba2c24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:27:38.616800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 634/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:27:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:27:38.639261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f222d-efde-48ff-a1a8-2bcb9a46ed1d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:38.651391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:38.751111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f222d-efde-48ff-a1a8-2bcb9a46ed1d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:38.784545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c77b6-4e85-49cf-8a52-2f0162e794bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:38.785123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c77b6-4e85-49cf-8a52-2f0162e794bd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:38.785409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c77b6-4e85-49cf-8a52-2f0162e794bd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:38.785690 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c77b6-4e85-49cf-8a52-2f0162e794bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:38.797845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c77b6-4e85-49cf-8a52-2f0162e794bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-207c77b6-4e85-49cf-8a52-2f0162e794bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:27:38.802871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:38.803847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:38.809651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 629/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:27:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:38.811163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a86f2-6c2e-46b1-87be-c43c9ba2c24f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:38.830490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:38.913198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a86f2-6c2e-46b1-87be-c43c9ba2c24f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:38.949958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ec3b9-810f-481e-88a4-b3d94d5db533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:38.949958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ec3b9-810f-481e-88a4-b3d94d5db533 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:38.949958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ec3b9-810f-481e-88a4-b3d94d5db533 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:38.949958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ec3b9-810f-481e-88a4-b3d94d5db533 None admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:38.951163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ec3b9-810f-481e-88a4-b3d94d5db533 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:38.951476 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ec3b9-810f-481e-88a4-b3d94d5db533 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:38.952851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ec3b9-810f-481e-88a4-b3d94d5db533 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:38.961692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 635/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:27:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:38.968021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c77b6-4e85-49cf-8a52-2f0162e794bd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:39.037323 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:39.069418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c77b6-4e85-49cf-8a52-2f0162e794bd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:39.102453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea741e2-7ed9-452a-a49d-6892799a5d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:39.103097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea741e2-7ed9-452a-a49d-6892799a5d92 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:39.103385 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea741e2-7ed9-452a-a49d-6892799a5d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:39.103662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea741e2-7ed9-452a-a49d-6892799a5d92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:39.113629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea741e2-7ed9-452a-a49d-6892799a5d92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:39.115053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea741e2-7ed9-452a-a49d-6892799a5d92 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9ea741e2-7ed9-452a-a49d-6892799a5d92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:39.129128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:39.130409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:39.131741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:39.131741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:39.131741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:39.131741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:39.131741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 630/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:39.136269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ec3b9-810f-481e-88a4-b3d94d5db533 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:39.217214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:39.235812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ec3b9-810f-481e-88a4-b3d94d5db533 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:39.268208 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad08c7-573f-46b5-bcaf-18e021efa7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:39.268812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad08c7-573f-46b5-bcaf-18e021efa7db None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:39.269093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad08c7-573f-46b5-bcaf-18e021efa7db None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:39.269367 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad08c7-573f-46b5-bcaf-18e021efa7db None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:39.270042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad08c7-573f-46b5-bcaf-18e021efa7db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:39.270348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad08c7-573f-46b5-bcaf-18e021efa7db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:39.271728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad08c7-573f-46b5-bcaf-18e021efa7db None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:39.284696 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 636/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:39.290302 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea741e2-7ed9-452a-a49d-6892799a5d92 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:39.385347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea741e2-7ed9-452a-a49d-6892799a5d92 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:39.404860 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:39.418490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cc3d7-60e4-4329-a7b4-724a374132d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:39.419129 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cc3d7-60e4-4329-a7b4-724a374132d0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:39.419439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cc3d7-60e4-4329-a7b4-724a374132d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:39.419716 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cc3d7-60e4-4329-a7b4-724a374132d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:39.427712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cc3d7-60e4-4329-a7b4-724a374132d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:39.429131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cc3d7-60e4-4329-a7b4-724a374132d0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-474cc3d7-60e4-4329-a7b4-724a374132d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:27:39.440423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:39.441500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:39.442614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:39.442614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:39.442614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:27:39.442614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:27:39.442614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:27:39.442614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:27:39.442614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:27:39.442614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:27:39.442614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:27:39.442614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:39.442614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 631/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:39.447241 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad08c7-573f-46b5-bcaf-18e021efa7db None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:39.543044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad08c7-573f-46b5-bcaf-18e021efa7db None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:39.576364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89a466-7f64-4ba4-8a2f-bcd48d0690fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:39.576949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89a466-7f64-4ba4-8a2f-bcd48d0690fc None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:39.577227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89a466-7f64-4ba4-8a2f-bcd48d0690fc None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:39.577501 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89a466-7f64-4ba4-8a2f-bcd48d0690fc None admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:39.578275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89a466-7f64-4ba4-8a2f-bcd48d0690fc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:39.578580 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89a466-7f64-4ba4-8a2f-bcd48d0690fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:39.580048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89a466-7f64-4ba4-8a2f-bcd48d0690fc None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:39.588280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:39.597104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 637/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Aug 1 07:27:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:39.602659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cc3d7-60e4-4329-a7b4-724a374132d0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:39.708686 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cc3d7-60e4-4329-a7b4-724a374132d0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:39.740159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b9be5-cc29-4e6b-936e-2edad5cacd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:39.740779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b9be5-cc29-4e6b-936e-2edad5cacd3d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:39.741060 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b9be5-cc29-4e6b-936e-2edad5cacd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:39.741336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b9be5-cc29-4e6b-936e-2edad5cacd3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:39.751957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b9be5-cc29-4e6b-936e-2edad5cacd3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:39.753379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b9be5-cc29-4e6b-936e-2edad5cacd3d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-717b9be5-cc29-4e6b-936e-2edad5cacd3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:39.767756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:39.768796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:39.769896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:39.769896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:39.769896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:39.769896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:39.769896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 632/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:39.774550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89a466-7f64-4ba4-8a2f-bcd48d0690fc None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:39.870750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89a466-7f64-4ba4-8a2f-bcd48d0690fc None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:39.903443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00411460-5d6b-479a-84d7-e6091786a0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:39.904089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00411460-5d6b-479a-84d7-e6091786a0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:39.904371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00411460-5d6b-479a-84d7-e6091786a0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:39.904643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00411460-5d6b-479a-84d7-e6091786a0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:39.905322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00411460-5d6b-479a-84d7-e6091786a0a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:39.905624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00411460-5d6b-479a-84d7-e6091786a0a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:39.906973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00411460-5d6b-479a-84d7-e6091786a0a7 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:39.917713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 638/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:39.924236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b9be5-cc29-4e6b-936e-2edad5cacd3d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:39.956227 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:40.023241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b9be5-cc29-4e6b-936e-2edad5cacd3d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:40.055258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa03c08-2c9e-453c-acca-31626d9e9f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:40.055875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa03c08-2c9e-453c-acca-31626d9e9f47 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:40.056155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa03c08-2c9e-453c-acca-31626d9e9f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:40.056426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa03c08-2c9e-453c-acca-31626d9e9f47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:40.063363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa03c08-2c9e-453c-acca-31626d9e9f47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:40.064785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa03c08-2c9e-453c-acca-31626d9e9f47 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0fa03c08-2c9e-453c-acca-31626d9e9f47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:27:40.075638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:40.076674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:40.077641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:27:40.077641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:27:40.077641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:27:40.077641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:27:40.077641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:27:40.077641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:27:40.077641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:27:40.077641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:40.077641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 633/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:27:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:40.082508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00411460-5d6b-479a-84d7-e6091786a0a7 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:40.139520 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:40.180346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00411460-5d6b-479a-84d7-e6091786a0a7 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:40.211849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f3d45-7cf5-4ff6-bd6c-85db395627fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:40.212453 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f3d45-7cf5-4ff6-bd6c-85db395627fd None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:40.212735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f3d45-7cf5-4ff6-bd6c-85db395627fd None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:40.213012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f3d45-7cf5-4ff6-bd6c-85db395627fd None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:40.213750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f3d45-7cf5-4ff6-bd6c-85db395627fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:40.214061 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f3d45-7cf5-4ff6-bd6c-85db395627fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:40.215509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f3d45-7cf5-4ff6-bd6c-85db395627fd None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:40.227834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 639/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:27:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:40.234885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa03c08-2c9e-453c-acca-31626d9e9f47 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:40.323283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:40.332066 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa03c08-2c9e-453c-acca-31626d9e9f47 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:40.365200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d663958-92f8-4955-bc80-2f1a9d932b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:40.365852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d663958-92f8-4955-bc80-2f1a9d932b3b None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:40.366139 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d663958-92f8-4955-bc80-2f1a9d932b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:40.366421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d663958-92f8-4955-bc80-2f1a9d932b3b None admin] PATH_INFO: `/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/928c5d1284b74f1291c031323bb03a20/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:40.368363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d663958-92f8-4955-bc80-2f1a9d932b3b None admin] RBAC: Authorizing `identity:create_grant(project_id=08ddf7d6f2424a85a741d017497b49c9, user_id=928c5d1284b74f1291c031323bb03a20, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:40.369803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d663958-92f8-4955-bc80-2f1a9d932b3b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:40.382062 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 634/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Aug 1 07:27:40 2022] PUT /identity/v3/projects/08ddf7d6f2424a85a741d017497b49c9/users/928c5d1284b74f1291c031323bb03a20/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:27:42.361640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe4755-3239-47e2-98fe-2a83bfed751a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:42.362133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe4755-3239-47e2-98fe-2a83bfed751a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:42.362133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe4755-3239-47e2-98fe-2a83bfed751a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:42.363534 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 640/1274] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:27:42.368432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5069b-bf12-4188-ac5a-b07868f86c33 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:42.368825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5069b-bf12-4188-ac5a-b07868f86c33 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:42.368991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5069b-bf12-4188-ac5a-b07868f86c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:42.387401 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9ca5069b-bf12-4188-ac5a-b07868f86c33 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:27:42.388684 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9ca5069b-bf12-4188-ac5a-b07868f86c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:27:42.451913 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 635/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:27:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:27:42.474637 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe4755-3239-47e2-98fe-2a83bfed751a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:42.573527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe4755-3239-47e2-98fe-2a83bfed751a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:42.590225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:42.608435 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54d6b2-f6fb-473e-a06d-186a956e1eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:42.609067 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54d6b2-f6fb-473e-a06d-186a956e1eac None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:42.609352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54d6b2-f6fb-473e-a06d-186a956e1eac None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:42.609632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54d6b2-f6fb-473e-a06d-186a956e1eac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:42.619458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54d6b2-f6fb-473e-a06d-186a956e1eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-4c54d6b2-f6fb-473e-a06d-186a956e1eac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 07:27:42.624568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:42.625651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:42.626860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 641/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Aug 1 07:27:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:42.631829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5069b-bf12-4188-ac5a-b07868f86c33 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:42.708193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:42.727427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5069b-bf12-4188-ac5a-b07868f86c33 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:42.761084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52825b7-562b-4856-93c2-379cb2d446cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:42.761686 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52825b7-562b-4856-93c2-379cb2d446cb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:42.761970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52825b7-562b-4856-93c2-379cb2d446cb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:42.762248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52825b7-562b-4856-93c2-379cb2d446cb None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:42.763524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52825b7-562b-4856-93c2-379cb2d446cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:42.764010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52825b7-562b-4856-93c2-379cb2d446cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:42.765376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52825b7-562b-4856-93c2-379cb2d446cb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:42.773079 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 636/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Aug 1 07:27:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:42.779667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54d6b2-f6fb-473e-a06d-186a956e1eac None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:42.877884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54d6b2-f6fb-473e-a06d-186a956e1eac None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:42.911528 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e6a2b-f836-4777-9b40-22484c011908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:42.912152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e6a2b-f836-4777-9b40-22484c011908 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:42.912438 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e6a2b-f836-4777-9b40-22484c011908 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:42.912721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e6a2b-f836-4777-9b40-22484c011908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:42.922463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e6a2b-f836-4777-9b40-22484c011908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:42.923937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e6a2b-f836-4777-9b40-22484c011908 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-bc5e6a2b-f836-4777-9b40-22484c011908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:42.938130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:42.939198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:42.940293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:42.940293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:42.940293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:42.940293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:42.940293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 642/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:42.944966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52825b7-562b-4856-93c2-379cb2d446cb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:43.044433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52825b7-562b-4856-93c2-379cb2d446cb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:43.076693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc45f0-f44d-4852-8005-7cc2975e2f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:43.077287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc45f0-f44d-4852-8005-7cc2975e2f03 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:43.077568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc45f0-f44d-4852-8005-7cc2975e2f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:43.077841 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc45f0-f44d-4852-8005-7cc2975e2f03 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:43.078513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc45f0-f44d-4852-8005-7cc2975e2f03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:43.078816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc45f0-f44d-4852-8005-7cc2975e2f03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:43.080196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc45f0-f44d-4852-8005-7cc2975e2f03 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:43.091981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 637/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:43.097673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e6a2b-f836-4777-9b40-22484c011908 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:43.127859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:43.196446 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e6a2b-f836-4777-9b40-22484c011908 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:43.228804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d16d52-bead-41dc-8a06-a73afb9b9343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:43.229430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d16d52-bead-41dc-8a06-a73afb9b9343 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:43.229711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d16d52-bead-41dc-8a06-a73afb9b9343 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:43.229985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d16d52-bead-41dc-8a06-a73afb9b9343 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:43.237587 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d16d52-bead-41dc-8a06-a73afb9b9343 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:43.239011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d16d52-bead-41dc-8a06-a73afb9b9343 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-30d16d52-bead-41dc-8a06-a73afb9b9343 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 07:27:43.250475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:43.251631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:43.252783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:43.252783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:43.252783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 07:27:43.252783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 07:27:43.252783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 07:27:43.252783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 07:27:43.252783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 07:27:43.252783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 07:27:43.252783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 07:27:43.252783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:43.252783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 643/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:43.257751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc45f0-f44d-4852-8005-7cc2975e2f03 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:43.261464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:43.355056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc45f0-f44d-4852-8005-7cc2975e2f03 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:43.389184 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80df9c4-2615-4ae3-acf3-d3839ea8f5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:43.389794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80df9c4-2615-4ae3-acf3-d3839ea8f5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:43.390080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80df9c4-2615-4ae3-acf3-d3839ea8f5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:43.390358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80df9c4-2615-4ae3-acf3-d3839ea8f5d1 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:43.391174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80df9c4-2615-4ae3-acf3-d3839ea8f5d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:43.391484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80df9c4-2615-4ae3-acf3-d3839ea8f5d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:43.392899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80df9c4-2615-4ae3-acf3-d3839ea8f5d1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:43.409547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 638/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Aug 1 07:27:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:43.415295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d16d52-bead-41dc-8a06-a73afb9b9343 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:43.502741 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:43.512533 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d16d52-bead-41dc-8a06-a73afb9b9343 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:43.545582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de430730-9334-4ded-9f78-1ac93fcd8d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:43.546179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de430730-9334-4ded-9f78-1ac93fcd8d66 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:43.546458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de430730-9334-4ded-9f78-1ac93fcd8d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:43.546734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de430730-9334-4ded-9f78-1ac93fcd8d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:43.612492 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de430730-9334-4ded-9f78-1ac93fcd8d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:43.613917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de430730-9334-4ded-9f78-1ac93fcd8d66 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:43.698749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:43.709041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 644/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Aug 1 07:27:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 07:27:43.714785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80df9c4-2615-4ae3-acf3-d3839ea8f5d1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:43.774919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:43.812523 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80df9c4-2615-4ae3-acf3-d3839ea8f5d1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:43.844571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7827e-b4ea-4811-903e-d1569436606e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:43.845189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7827e-b4ea-4811-903e-d1569436606e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:43.845473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7827e-b4ea-4811-903e-d1569436606e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:43.845750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7827e-b4ea-4811-903e-d1569436606e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:43.855358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7827e-b4ea-4811-903e-d1569436606e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:43.857318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7827e-b4ea-4811-903e-d1569436606e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-55e7827e-b4ea-4811-903e-d1569436606e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 07:27:43.871364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:43.873110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:43.874306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 07:27:43.874306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 07:27:43.874306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 07:27:43.874306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:43.874306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 639/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:43.878613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de430730-9334-4ded-9f78-1ac93fcd8d66 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:43.974773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de430730-9334-4ded-9f78-1ac93fcd8d66 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:44.008486 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18c657-9dc8-4533-837f-2d67c52592c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:44.009098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18c657-9dc8-4533-837f-2d67c52592c9 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:44.009381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18c657-9dc8-4533-837f-2d67c52592c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:44.009654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18c657-9dc8-4533-837f-2d67c52592c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:44.010331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18c657-9dc8-4533-837f-2d67c52592c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:44.010635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18c657-9dc8-4533-837f-2d67c52592c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:44.012028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18c657-9dc8-4533-837f-2d67c52592c9 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:44.023186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 645/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Aug 1 07:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:44.029831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7827e-b4ea-4811-903e-d1569436606e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:44.129235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7827e-b4ea-4811-903e-d1569436606e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:44.142856 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:44.163862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333c97a-3e65-4c3d-b8d7-3b134cc12879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:44.164482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333c97a-3e65-4c3d-b8d7-3b134cc12879 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:44.164764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333c97a-3e65-4c3d-b8d7-3b134cc12879 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:44.165042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333c97a-3e65-4c3d-b8d7-3b134cc12879 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:44.172604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333c97a-3e65-4c3d-b8d7-3b134cc12879 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:44.174031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333c97a-3e65-4c3d-b8d7-3b134cc12879 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a333c97a-3e65-4c3d-b8d7-3b134cc12879 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 07:27:44.186361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:44.187446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:44.188483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 07:27:44.188483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 07:27:44.188483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 07:27:44.188483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 07:27:44.188483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 07:27:44.188483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 07:27:44.188483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 07:27:44.188483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Aug 01 07:27:44.188483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 640/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Aug 1 07:27:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:44.193352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18c657-9dc8-4533-837f-2d67c52592c9 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:44.243466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:44.304620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18c657-9dc8-4533-837f-2d67c52592c9 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:44.336701 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40b27e-b386-4d63-8d6f-f338869bccb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:44.337314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40b27e-b386-4d63-8d6f-f338869bccb5 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:44.337594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40b27e-b386-4d63-8d6f-f338869bccb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:44.337870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40b27e-b386-4d63-8d6f-f338869bccb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:44.338598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40b27e-b386-4d63-8d6f-f338869bccb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:44.338904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40b27e-b386-4d63-8d6f-f338869bccb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:44.340345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40b27e-b386-4d63-8d6f-f338869bccb5 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:44.352583 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 646/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Aug 1 07:27:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:44.358787 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333c97a-3e65-4c3d-b8d7-3b134cc12879 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:44.454512 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333c97a-3e65-4c3d-b8d7-3b134cc12879 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:44.486837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e6129-d601-4ed8-bf0f-c1a1226fc7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:44.487569 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e6129-d601-4ed8-bf0f-c1a1226fc7cb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:44.487850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e6129-d601-4ed8-bf0f-c1a1226fc7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:44.488125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e6129-d601-4ed8-bf0f-c1a1226fc7cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:44.488896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e6129-d601-4ed8-bf0f-c1a1226fc7cb None admin] RBAC: Adding query filter params (role.id=69aa06056979475d8b9fe3bcd9298f02, scope.project.id=08ddf7d6f2424a85a741d017497b49c9, user.id=928c5d1284b74f1291c031323bb03a20) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:44.489203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e6129-d601-4ed8-bf0f-c1a1226fc7cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:44.490601 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e6129-d601-4ed8-bf0f-c1a1226fc7cb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:44.496185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 641/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Aug 1 07:27:44 2022] GET /identity/v3/role_assignments?user.id=928c5d1284b74f1291c031323bb03a20&scope.project.id=08ddf7d6f2424a85a741d017497b49c9&role.id=69aa06056979475d8b9fe3bcd9298f02 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:46.504556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bebfc-0808-4d5e-bd96-63ebbd965ec3 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:46.505401 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bebfc-0808-4d5e-bd96-63ebbd965ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:46.505401 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bebfc-0808-4d5e-bd96-63ebbd965ec3 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:46.506692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 647/1288] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:27:46.511692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d2f8-cc4a-45a3-85b6-1eb98da70718 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:46.512135 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d2f8-cc4a-45a3-85b6-1eb98da70718 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:46.512264 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d2f8-cc4a-45a3-85b6-1eb98da70718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:46.530573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8a30d2f8-cc4a-45a3-85b6-1eb98da70718 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:27:46.531806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8a30d2f8-cc4a-45a3-85b6-1eb98da70718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:27:46.561533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:46.598382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 642/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:27:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:27:46.620506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bebfc-0808-4d5e-bd96-63ebbd965ec3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:46.733682 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bebfc-0808-4d5e-bd96-63ebbd965ec3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:46.744050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:46.767939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad536c89-eb09-4fc7-9455-397d5e1d9b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:46.768512 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad536c89-eb09-4fc7-9455-397d5e1d9b64 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:46.768791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad536c89-eb09-4fc7-9455-397d5e1d9b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:46.769063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad536c89-eb09-4fc7-9455-397d5e1d9b64 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:46.769679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad536c89-eb09-4fc7-9455-397d5e1d9b64 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:46.771021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad536c89-eb09-4fc7-9455-397d5e1d9b64 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-ad536c89-eb09-4fc7-9455-397d5e1d9b64 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:46.779660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:46.780912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:46.782031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 648/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Aug 1 07:27:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:46.786749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d2f8-cc4a-45a3-85b6-1eb98da70718 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:46.881112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d2f8-cc4a-45a3-85b6-1eb98da70718 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:46.913202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9561cb-7fab-4d98-b076-d01340100c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:46.913798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9561cb-7fab-4d98-b076-d01340100c72 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:46.914076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9561cb-7fab-4d98-b076-d01340100c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:46.914354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9561cb-7fab-4d98-b076-d01340100c72 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:46.915018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9561cb-7fab-4d98-b076-d01340100c72 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:46.915465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9561cb-7fab-4d98-b076-d01340100c72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:46.916817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9561cb-7fab-4d98-b076-d01340100c72 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:46.921518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 643/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Mon Aug 1 07:27:46 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:46.927134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad536c89-eb09-4fc7-9455-397d5e1d9b64 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:47.025201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad536c89-eb09-4fc7-9455-397d5e1d9b64 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:47.057129 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a70f77-805e-4913-a3ad-fb97bc91a6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:47.057737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a70f77-805e-4913-a3ad-fb97bc91a6db None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:47.058022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a70f77-805e-4913-a3ad-fb97bc91a6db None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:47.058303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a70f77-805e-4913-a3ad-fb97bc91a6db None admin] PATH_INFO: `/v3/services` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:47.058975 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a70f77-805e-4913-a3ad-fb97bc91a6db None admin] RBAC: Adding query filter params (type=placement) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:47.059318 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a70f77-805e-4913-a3ad-fb97bc91a6db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:47.061199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a70f77-805e-4913-a3ad-fb97bc91a6db None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:47.065577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 649/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Mon Aug 1 07:27:46 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:49.078796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98afeb-d8c4-40b3-b5f0-2bfa71f40a87 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:49.079333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98afeb-d8c4-40b3-b5f0-2bfa71f40a87 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:49.079438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98afeb-d8c4-40b3-b5f0-2bfa71f40a87 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:49.080731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 644/1293] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:27:49.085748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0b746-0714-4914-916c-9036d049cfa3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:49.086127 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0b746-0714-4914-916c-9036d049cfa3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:49.086307 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0b746-0714-4914-916c-9036d049cfa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:49.104406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7fb0b746-0714-4914-916c-9036d049cfa3 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:27:49.105576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7fb0b746-0714-4914-916c-9036d049cfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:27:49.133855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:49.170550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 650/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:27:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:27:49.193483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98afeb-d8c4-40b3-b5f0-2bfa71f40a87 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:49.225203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:49.295583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98afeb-d8c4-40b3-b5f0-2bfa71f40a87 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:49.327800 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9debdc8-5113-4076-b6d6-3712dcbf101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:49.328405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9debdc8-5113-4076-b6d6-3712dcbf101c None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:49.328686 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9debdc8-5113-4076-b6d6-3712dcbf101c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:49.328958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9debdc8-5113-4076-b6d6-3712dcbf101c None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:49.329626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9debdc8-5113-4076-b6d6-3712dcbf101c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:49.330979 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9debdc8-5113-4076-b6d6-3712dcbf101c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:49.344976 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 645/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Aug 1 07:27:49 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:27:51.374112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076993a-2a2f-4910-859a-1aa4a5a0be1e None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:51.374638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076993a-2a2f-4910-859a-1aa4a5a0be1e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:51.374638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076993a-2a2f-4910-859a-1aa4a5a0be1e None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:51.375974 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 651/1296] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:27:51.381326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf66e20-5ee6-47ae-ab87-11677ff233ac None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:51.381760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf66e20-5ee6-47ae-ab87-11677ff233ac None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:51.381887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf66e20-5ee6-47ae-ab87-11677ff233ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:51.400196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ebf66e20-5ee6-47ae-ab87-11677ff233ac None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:27:51.401381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ebf66e20-5ee6-47ae-ab87-11677ff233ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:27:51.472472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 646/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:27:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:27:51.494445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076993a-2a2f-4910-859a-1aa4a5a0be1e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:51.600963 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076993a-2a2f-4910-859a-1aa4a5a0be1e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:51.633978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b5105-5239-4dad-88cc-4467af084988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:51.634544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b5105-5239-4dad-88cc-4467af084988 None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:51.634829 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b5105-5239-4dad-88cc-4467af084988 None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:51.635134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b5105-5239-4dad-88cc-4467af084988 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:51.635756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b5105-5239-4dad-88cc-4467af084988 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:51.637120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b5105-5239-4dad-88cc-4467af084988 None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-9e2b5105-5239-4dad-88cc-4467af084988 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:51.646577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:51.647773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:51.648734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 652/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Aug 1 07:27:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:51.654444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf66e20-5ee6-47ae-ab87-11677ff233ac None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:51.675636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:51.762904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf66e20-5ee6-47ae-ab87-11677ff233ac None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:51.796794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d1f00-2f05-4e34-9db0-74996ae58e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:51.797400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d1f00-2f05-4e34-9db0-74996ae58e72 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:51.797683 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d1f00-2f05-4e34-9db0-74996ae58e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:51.797960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d1f00-2f05-4e34-9db0-74996ae58e72 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:51.798629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d1f00-2f05-4e34-9db0-74996ae58e72 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:51.798933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d1f00-2f05-4e34-9db0-74996ae58e72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:51.800337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d1f00-2f05-4e34-9db0-74996ae58e72 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:51.805303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 647/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Mon Aug 1 07:27:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:27:51.812242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b5105-5239-4dad-88cc-4467af084988 None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:51.843938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:51.910798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b5105-5239-4dad-88cc-4467af084988 None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:51.943424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a6892-b830-4e84-b05e-69c9da5dbacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:51.944027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a6892-b830-4e84-b05e-69c9da5dbacb None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:51.944308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a6892-b830-4e84-b05e-69c9da5dbacb None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:51.944585 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a6892-b830-4e84-b05e-69c9da5dbacb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:51.945370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a6892-b830-4e84-b05e-69c9da5dbacb None admin] RBAC: Adding query filter params (interface=public, service_id=1c0530be6d88421aa04e3e7a6073dc98, region_id=RegionOne) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:51.945679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a6892-b830-4e84-b05e-69c9da5dbacb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:51.947051 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a6892-b830-4e84-b05e-69c9da5dbacb None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:51.952323 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 653/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Aug 1 07:27:51 2022] GET /identity/v3/endpoints?service_id=1c0530be6d88421aa04e3e7a6073dc98&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:27:51.957890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d1f00-2f05-4e34-9db0-74996ae58e72 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:52.037574 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:52.055831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d1f00-2f05-4e34-9db0-74996ae58e72 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:52.088651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a183825-9b76-48e7-80b7-4c26b9b7e156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:52.089258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a183825-9b76-48e7-80b7-4c26b9b7e156 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:52.089549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a183825-9b76-48e7-80b7-4c26b9b7e156 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:52.089830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a183825-9b76-48e7-80b7-4c26b9b7e156 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:52.090659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a183825-9b76-48e7-80b7-4c26b9b7e156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:52.092102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a183825-9b76-48e7-80b7-4c26b9b7e156 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:52.098779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 648/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Aug 1 07:27:51 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:27:54.063386 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08d7f4-286f-4bf5-8801-96113374f402 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:54.063919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08d7f4-286f-4bf5-8801-96113374f402 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:54.063919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08d7f4-286f-4bf5-8801-96113374f402 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:54.065276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 654/1302] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:27:54.070231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a6311-ce98-4621-9568-55cc2e0fa751 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:54.070636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a6311-ce98-4621-9568-55cc2e0fa751 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:54.070817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a6311-ce98-4621-9568-55cc2e0fa751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:54.088989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e30a6311-ce98-4621-9568-55cc2e0fa751 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:27:54.090193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e30a6311-ce98-4621-9568-55cc2e0fa751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:27:54.155130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 649/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:27:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 07:27:54.177493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08d7f4-286f-4bf5-8801-96113374f402 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:54.269334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:54.276254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08d7f4-286f-4bf5-8801-96113374f402 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:54.310006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589bd7b-186b-409c-8a19-70a5615d7c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:54.310630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589bd7b-186b-409c-8a19-70a5615d7c1d None admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:54.310912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589bd7b-186b-409c-8a19-70a5615d7c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:54.311219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589bd7b-186b-409c-8a19-70a5615d7c1d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:54.311839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589bd7b-186b-409c-8a19-70a5615d7c1d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:54.313200 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589bd7b-186b-409c-8a19-70a5615d7c1d None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application [None req-3589bd7b-186b-409c-8a19-70a5615d7c1d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 07:27:54.321885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 07:27:54.323053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application with Lock( Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self._enter() Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application return self.creator() Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application created_value = creator( Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: ERROR keystone.server.flask.application Aug 01 07:27:54.323983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 655/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Aug 1 07:27:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 07:27:54.329050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a6311-ce98-4621-9568-55cc2e0fa751 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:54.403932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:27:54.426457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a6311-ce98-4621-9568-55cc2e0fa751 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:54.458897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3a5f5-e6e3-4580-9537-b52cdbc22fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:54.459547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3a5f5-e6e3-4580-9537-b52cdbc22fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:54.459834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3a5f5-e6e3-4580-9537-b52cdbc22fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:54.460113 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3a5f5-e6e3-4580-9537-b52cdbc22fa6 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:54.460787 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3a5f5-e6e3-4580-9537-b52cdbc22fa6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:27:54.461097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3a5f5-e6e3-4580-9537-b52cdbc22fa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:54.462461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3a5f5-e6e3-4580-9537-b52cdbc22fa6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:54.468049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 650/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Mon Aug 1 07:27:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:27:54.473767 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589bd7b-186b-409c-8a19-70a5615d7c1d None admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:27:54.569999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589bd7b-186b-409c-8a19-70a5615d7c1d None admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:27:54.602149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f853243-02b5-4234-a9e9-5a67bfadb26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:27:54.602748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f853243-02b5-4234-a9e9-5a67bfadb26b None admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:27:54.603028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f853243-02b5-4234-a9e9-5a67bfadb26b None admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:27:54.603334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f853243-02b5-4234-a9e9-5a67bfadb26b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:27:54.604020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f853243-02b5-4234-a9e9-5a67bfadb26b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:27:54.605367 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f853243-02b5-4234-a9e9-5a67bfadb26b None admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:27:54.624726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 656/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:27:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:28:04.464480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=30 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:28:04.620740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:28:57.501624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ba929-704c-4331-b50e-028458b98672 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:28:57.502663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ba929-704c-4331-b50e-028458b98672 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:28:57.503497 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ba929-704c-4331-b50e-028458b98672 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:28:57.515723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 651/1307] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:28:57 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:28:57.518608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbb738-ab6c-4b83-b57f-d92f5bb1c2df None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:28:57.519480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbb738-ab6c-4b83-b57f-d92f5bb1c2df None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:28:57.519969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbb738-ab6c-4b83-b57f-d92f5bb1c2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:28:57.601509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-53cbb738-ab6c-4b83-b57f-d92f5bb1c2df None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:28:57.603706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-53cbb738-ab6c-4b83-b57f-d92f5bb1c2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:28:57.736596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 657/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:28:57.807469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e46df8-6161-4a36-a2f6-e26ad67ec26e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:28:57.808233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e46df8-6161-4a36-a2f6-e26ad67ec26e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:28:57.808893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e46df8-6161-4a36-a2f6-e26ad67ec26e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:28:57.810957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 652/1309] 10.222.0.40 () {54 vars in 965 bytes} [Mon Aug 1 07:28:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:28:57.817228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98235b-118f-4cc4-9b4d-efedbc28a8c8 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:28:57.817561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98235b-118f-4cc4-9b4d-efedbc28a8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:28:57.817780 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98235b-118f-4cc4-9b4d-efedbc28a8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:28:57.978214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dd98235b-118f-4cc4-9b4d-efedbc28a8c8 None None] MFA Rules not processed for user `138ede2854894a3b995435980965911c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:28:57.979854 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dd98235b-118f-4cc4-9b4d-efedbc28a8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:28:58.033090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:28:58.169215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 658/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Mon Aug 1 07:28:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:28:58.178332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e46df8-6161-4a36-a2f6-e26ad67ec26e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:28:58.180768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-38e46df8-6161-4a36-a2f6-e26ad67ec26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:28:58.441479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e46df8-6161-4a36-a2f6-e26ad67ec26e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:28:58.555719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6522a0-8f6a-4900-b40e-2b7ad6e22681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:28:58.556491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6522a0-8f6a-4900-b40e-2b7ad6e22681 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:28:58.556723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6522a0-8f6a-4900-b40e-2b7ad6e22681 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:28:58.556999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6522a0-8f6a-4900-b40e-2b7ad6e22681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:28:58.558684 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2e6522a0-8f6a-4900-b40e-2b7ad6e22681 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:28:58.630176 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:28:58.694713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6522a0-8f6a-4900-b40e-2b7ad6e22681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:28:58.696444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6522a0-8f6a-4900-b40e-2b7ad6e22681 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:28:58.720242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 653/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:01.132725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fd682-d4f5-415d-b06b-6af994a43fbc None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:01.133729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fd682-d4f5-415d-b06b-6af994a43fbc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:01.133808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fd682-d4f5-415d-b06b-6af994a43fbc None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:01.135184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 659/1312] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:01.140539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8f348-b298-4782-8410-fd318a01add9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:01.140974 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8f348-b298-4782-8410-fd318a01add9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:01.141100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8f348-b298-4782-8410-fd318a01add9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:01.225152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8ef8f348-b298-4782-8410-fd318a01add9 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:01.226321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8ef8f348-b298-4782-8410-fd318a01add9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:01.332899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 654/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:01.385191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be9c86-33ba-463c-a0ae-dad6243ed46a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:01.385480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be9c86-33ba-463c-a0ae-dad6243ed46a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:01.385766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be9c86-33ba-463c-a0ae-dad6243ed46a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:01.387135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 660/1314] 10.222.0.40 () {54 vars in 965 bytes} [Mon Aug 1 07:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:01.393255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ff52d-a84c-4652-baed-08e29d03c999 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:01.393548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ff52d-a84c-4652-baed-08e29d03c999 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:01.393829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ff52d-a84c-4652-baed-08e29d03c999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:01.406246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:01.469410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2a4ff52d-a84c-4652-baed-08e29d03c999 None None] MFA Rules not processed for user `138ede2854894a3b995435980965911c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:01.470636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2a4ff52d-a84c-4652-baed-08e29d03c999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:01.550665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 655/1315] 10.222.0.40 () {58 vars in 1062 bytes} [Mon Aug 1 07:29:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:29:01.559758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be9c86-33ba-463c-a0ae-dad6243ed46a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:01.561385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-05be9c86-33ba-463c-a0ae-dad6243ed46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:01.670332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be9c86-33ba-463c-a0ae-dad6243ed46a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:01.743893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:01.746903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5304f8d-4207-4017-b855-5ac5eddcebb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:01.747664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5304f8d-4207-4017-b855-5ac5eddcebb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:01.747982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5304f8d-4207-4017-b855-5ac5eddcebb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:01.748422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5304f8d-4207-4017-b855-5ac5eddcebb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:01.750051 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b5304f8d-4207-4017-b855-5ac5eddcebb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:01.859324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5304f8d-4207-4017-b855-5ac5eddcebb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:01.860687 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5304f8d-4207-4017-b855-5ac5eddcebb6 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:01.879929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 661/1316] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:04.430998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36879618-70ea-49d5-955b-5a830e68d6a7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:04.431542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36879618-70ea-49d5-955b-5a830e68d6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:04.431660 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36879618-70ea-49d5-955b-5a830e68d6a7 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:04.432969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 656/1317] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:04.438192 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22d86a-7ff5-4595-94ce-7bd7b4a77620 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:04.438569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22d86a-7ff5-4595-94ce-7bd7b4a77620 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:04.438729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22d86a-7ff5-4595-94ce-7bd7b4a77620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:04.502517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2e22d86a-7ff5-4595-94ce-7bd7b4a77620 None None] MFA Rules not processed for user `5b4524f7f2f6410e983dadf2d798c568`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:04.504012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2e22d86a-7ff5-4595-94ce-7bd7b4a77620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:04.554127 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:04.598716 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 662/1318] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:04 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:04.638353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36879618-70ea-49d5-955b-5a830e68d6a7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:04.639928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-36879618-70ea-49d5-955b-5a830e68d6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:04.746939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36879618-70ea-49d5-955b-5a830e68d6a7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:04.749728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:04.819367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18f037-ca7c-405c-98ba-6803bebfd008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:04.819980 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18f037-ca7c-405c-98ba-6803bebfd008 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:04.820783 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18f037-ca7c-405c-98ba-6803bebfd008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:04.821057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18f037-ca7c-405c-98ba-6803bebfd008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:04.822680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0c18f037-ca7c-405c-98ba-6803bebfd008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:04.959047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18f037-ca7c-405c-98ba-6803bebfd008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:04.960508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18f037-ca7c-405c-98ba-6803bebfd008 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:04.983831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 657/1319] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:04 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:07.580786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38ceb1-858f-402f-818c-17f87b50a921 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:07.581356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38ceb1-858f-402f-818c-17f87b50a921 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:07.581356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38ceb1-858f-402f-818c-17f87b50a921 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:07.582685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 663/1320] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:29:07.588506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd071f03-7006-494a-b0b9-997b51498a09 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:07.588934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd071f03-7006-494a-b0b9-997b51498a09 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:07.589071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd071f03-7006-494a-b0b9-997b51498a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:07.630425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:07.658714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cd071f03-7006-494a-b0b9-997b51498a09 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:07.659976 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cd071f03-7006-494a-b0b9-997b51498a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:07.766975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 658/1321] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:07.810605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38ceb1-858f-402f-818c-17f87b50a921 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:07.812203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bb38ceb1-858f-402f-818c-17f87b50a921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:07.917448 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38ceb1-858f-402f-818c-17f87b50a921 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:07.994951 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b66d4f-176d-4bfb-b579-dce985910b72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:07.995599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b66d4f-176d-4bfb-b579-dce985910b72 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:07.995888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b66d4f-176d-4bfb-b579-dce985910b72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:07.996170 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b66d4f-176d-4bfb-b579-dce985910b72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:07.997834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-90b66d4f-176d-4bfb-b579-dce985910b72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:08.005509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:08.106656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b66d4f-176d-4bfb-b579-dce985910b72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:08.108133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b66d4f-176d-4bfb-b579-dce985910b72 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:08.126681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 664/1322] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:10.992443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03e95d-fde1-4700-91fb-e3b1a39c6ba4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:10.993085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03e95d-fde1-4700-91fb-e3b1a39c6ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:10.993085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03e95d-fde1-4700-91fb-e3b1a39c6ba4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:10.994351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 659/1323] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:10.999513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0faef0-94d0-400b-abfb-8c402c9bd71e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:10.999931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0faef0-94d0-400b-abfb-8c402c9bd71e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:11.000179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0faef0-94d0-400b-abfb-8c402c9bd71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:11.068507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3e0faef0-94d0-400b-abfb-8c402c9bd71e None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:11.070485 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3e0faef0-94d0-400b-abfb-8c402c9bd71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:11.128644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:11.168017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 665/1324] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:11.205565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03e95d-fde1-4700-91fb-e3b1a39c6ba4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:11.207206 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5b03e95d-fde1-4700-91fb-e3b1a39c6ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:11.321783 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03e95d-fde1-4700-91fb-e3b1a39c6ba4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:11.335047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:11.399375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09ba62-e25e-498c-a340-634d1242e229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:11.399999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09ba62-e25e-498c-a340-634d1242e229 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:11.400283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09ba62-e25e-498c-a340-634d1242e229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:11.400567 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09ba62-e25e-498c-a340-634d1242e229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:11.402191 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e09ba62-e25e-498c-a340-634d1242e229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:11.509331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09ba62-e25e-498c-a340-634d1242e229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:11.510788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09ba62-e25e-498c-a340-634d1242e229 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:11.529032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 660/1325] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:13.619245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df80ab-2caf-4b0b-9db9-e2e17dc4fc37 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:13.619771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df80ab-2caf-4b0b-9db9-e2e17dc4fc37 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:13.619771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df80ab-2caf-4b0b-9db9-e2e17dc4fc37 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:13.621154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 666/1326] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:13.626211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c0dcc-5623-4524-aaa7-dd24abc9b1ed None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:13.626616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c0dcc-5623-4524-aaa7-dd24abc9b1ed None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:13.626778 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c0dcc-5623-4524-aaa7-dd24abc9b1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:13.676853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:13.695987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c97c0dcc-5623-4524-aaa7-dd24abc9b1ed None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:13.697202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c97c0dcc-5623-4524-aaa7-dd24abc9b1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:13.792715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 661/1327] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:13.828475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df80ab-2caf-4b0b-9db9-e2e17dc4fc37 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:13.829987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-39df80ab-2caf-4b0b-9db9-e2e17dc4fc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:13.934897 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df80ab-2caf-4b0b-9db9-e2e17dc4fc37 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:14.011101 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75df00-77c3-46b9-8397-f0bca4ab78f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:14.011720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75df00-77c3-46b9-8397-f0bca4ab78f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:14.011999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75df00-77c3-46b9-8397-f0bca4ab78f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:14.012271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75df00-77c3-46b9-8397-f0bca4ab78f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:14.013873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8d75df00-77c3-46b9-8397-f0bca4ab78f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:14.030554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:14.125973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75df00-77c3-46b9-8397-f0bca4ab78f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:14.127748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75df00-77c3-46b9-8397-f0bca4ab78f2 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:14.146912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 667/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:16.184066 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b10afd-7201-433b-b2c8-5ed94611bb45 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:16.184630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b10afd-7201-433b-b2c8-5ed94611bb45 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:16.184630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b10afd-7201-433b-b2c8-5ed94611bb45 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:16.186038 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 662/1329] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:16.191928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e54fd-2678-4355-ab58-c068701145e9 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:16.192447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e54fd-2678-4355-ab58-c068701145e9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:16.192535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e54fd-2678-4355-ab58-c068701145e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:16.262381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1f8e54fd-2678-4355-ab58-c068701145e9 None None] MFA Rules not processed for user `5b4524f7f2f6410e983dadf2d798c568`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:16.263535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1f8e54fd-2678-4355-ab58-c068701145e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:16.328054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:16.360031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 668/1330] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:16 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:16.398433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b10afd-7201-433b-b2c8-5ed94611bb45 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:16.400010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-54b10afd-7201-433b-b2c8-5ed94611bb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:16.509564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b10afd-7201-433b-b2c8-5ed94611bb45 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:16.530036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:16.586551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76a58d-0e03-4e03-9df7-232b98cb52ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:16.587262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a58d-0e03-4e03-9df7-232b98cb52ab service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:16.587549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a58d-0e03-4e03-9df7-232b98cb52ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:16.587827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a58d-0e03-4e03-9df7-232b98cb52ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:16.589448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ad76a58d-0e03-4e03-9df7-232b98cb52ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:16.705839 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a58d-0e03-4e03-9df7-232b98cb52ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:16.707326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a58d-0e03-4e03-9df7-232b98cb52ab service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:16.725188 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 663/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:19.025160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e280ba-34fb-404b-88c0-41cb514911c3 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:19.025756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e280ba-34fb-404b-88c0-41cb514911c3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:19.025756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e280ba-34fb-404b-88c0-41cb514911c3 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:19.027092 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 669/1332] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:19.032215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654ef0b-7655-425f-b687-a7f57311c948 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:19.032655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654ef0b-7655-425f-b687-a7f57311c948 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:19.032756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654ef0b-7655-425f-b687-a7f57311c948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:19.095985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:19.098777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7654ef0b-7655-425f-b687-a7f57311c948 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:19.100674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7654ef0b-7655-425f-b687-a7f57311c948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:19.198769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 664/1333] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:19.234222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e280ba-34fb-404b-88c0-41cb514911c3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:19.235781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-41e280ba-34fb-404b-88c0-41cb514911c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:19.337876 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e280ba-34fb-404b-88c0-41cb514911c3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:19.409558 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed75fa0-f173-480a-8b80-3e91ce7776b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:19.410173 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed75fa0-f173-480a-8b80-3e91ce7776b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:19.410455 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed75fa0-f173-480a-8b80-3e91ce7776b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:19.410725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed75fa0-f173-480a-8b80-3e91ce7776b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:19.412350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4ed75fa0-f173-480a-8b80-3e91ce7776b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:19.439004 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:19.520010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed75fa0-f173-480a-8b80-3e91ce7776b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:19.521360 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed75fa0-f173-480a-8b80-3e91ce7776b3 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:19.539767 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 670/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:21.602747 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9ac2-e3e1-4ce4-a6f7-99aed15df649 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:21.603347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9ac2-e3e1-4ce4-a6f7-99aed15df649 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:21.603347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9ac2-e3e1-4ce4-a6f7-99aed15df649 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:21.604749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 665/1335] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:21.609772 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0789a6b-865e-429b-be36-8d807145468f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:21.610165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0789a6b-865e-429b-be36-8d807145468f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:21.610329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0789a6b-865e-429b-be36-8d807145468f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:21.676073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c0789a6b-865e-429b-be36-8d807145468f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:21.677274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c0789a6b-865e-429b-be36-8d807145468f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:21.750132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:21.774503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 671/1336] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:21.813378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9ac2-e3e1-4ce4-a6f7-99aed15df649 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:21.814911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bdfd9ac2-e3e1-4ce4-a6f7-99aed15df649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:21.916693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9ac2-e3e1-4ce4-a6f7-99aed15df649 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:21.944476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:21.989940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734dc4c-d73a-4b09-8c2f-66815c0f9828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:21.990568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734dc4c-d73a-4b09-8c2f-66815c0f9828 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:21.990923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734dc4c-d73a-4b09-8c2f-66815c0f9828 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:21.991232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734dc4c-d73a-4b09-8c2f-66815c0f9828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:21.992847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a734dc4c-d73a-4b09-8c2f-66815c0f9828 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:22.100012 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734dc4c-d73a-4b09-8c2f-66815c0f9828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:22.101455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734dc4c-d73a-4b09-8c2f-66815c0f9828 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:22.118929 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 666/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:24.454725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155cb26-62ae-449e-940d-e698cd4d6a2d None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:24.455276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155cb26-62ae-449e-940d-e698cd4d6a2d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:24.455373 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155cb26-62ae-449e-940d-e698cd4d6a2d None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:24.456656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 672/1338] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:24.462557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ab737-d815-4cbc-846d-2d437e8196e2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:24.463021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ab737-d815-4cbc-846d-2d437e8196e2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:24.463141 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ab737-d815-4cbc-846d-2d437e8196e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:24.529176 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8e7ab737-d815-4cbc-846d-2d437e8196e2 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:24.530388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8e7ab737-d815-4cbc-846d-2d437e8196e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:24.539336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:24.625661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 667/1339] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:24.661749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155cb26-62ae-449e-940d-e698cd4d6a2d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:24.664151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4155cb26-62ae-449e-940d-e698cd4d6a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:24.768245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155cb26-62ae-449e-940d-e698cd4d6a2d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:24.841240 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00ca1c-14f4-4869-abab-0b207fcc2ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:24.841860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00ca1c-14f4-4869-abab-0b207fcc2ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:24.842150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00ca1c-14f4-4869-abab-0b207fcc2ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:24.842433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00ca1c-14f4-4869-abab-0b207fcc2ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:24.844074 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ed00ca1c-14f4-4869-abab-0b207fcc2ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:24.878868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:24.951643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00ca1c-14f4-4869-abab-0b207fcc2ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:24.953066 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00ca1c-14f4-4869-abab-0b207fcc2ee4 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:24.971666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 673/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:28.916525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cd5b2-92fa-4be1-bb53-c79bffeccae6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:28.917042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cd5b2-92fa-4be1-bb53-c79bffeccae6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:28.917042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cd5b2-92fa-4be1-bb53-c79bffeccae6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:28.918383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 668/1341] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:28.923513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d14a9e-cd34-40bf-931d-6507f1d7fc7f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:28.923936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d14a9e-cd34-40bf-931d-6507f1d7fc7f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:28.924116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d14a9e-cd34-40bf-931d-6507f1d7fc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:28.993852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-92d14a9e-cd34-40bf-931d-6507f1d7fc7f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:28.994999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-92d14a9e-cd34-40bf-931d-6507f1d7fc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:29.084050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:29.096555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 674/1342] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:29.136933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cd5b2-92fa-4be1-bb53-c79bffeccae6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:29.138558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-127cd5b2-92fa-4be1-bb53-c79bffeccae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:29.246200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cd5b2-92fa-4be1-bb53-c79bffeccae6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:29.285843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:29.324131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a2187c-a50d-4636-9eb8-ea2920a3cf57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:29.324754 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2187c-a50d-4636-9eb8-ea2920a3cf57 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:29.325036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2187c-a50d-4636-9eb8-ea2920a3cf57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:29.325313 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2187c-a50d-4636-9eb8-ea2920a3cf57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:29.326927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d2a2187c-a50d-4636-9eb8-ea2920a3cf57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:29.434456 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a2187c-a50d-4636-9eb8-ea2920a3cf57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:29.435920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a2187c-a50d-4636-9eb8-ea2920a3cf57 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:29.457048 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 669/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:32.133243 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bddf4e-6a4c-4c2d-81ce-932e6b75ad0e None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:32.133797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bddf4e-6a4c-4c2d-81ce-932e6b75ad0e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:32.133797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bddf4e-6a4c-4c2d-81ce-932e6b75ad0e None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:32.135244 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 675/1344] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:32.140687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204413c-87bb-429c-83ec-b5e9a4640d88 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:32.141130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204413c-87bb-429c-83ec-b5e9a4640d88 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:32.141262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204413c-87bb-429c-83ec-b5e9a4640d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:32.208775 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8204413c-87bb-429c-83ec-b5e9a4640d88 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:32.210019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8204413c-87bb-429c-83ec-b5e9a4640d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:32.227775 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:32.308346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 670/1345] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:32.346013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bddf4e-6a4c-4c2d-81ce-932e6b75ad0e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:32.347685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-25bddf4e-6a4c-4c2d-81ce-932e6b75ad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:32.452325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bddf4e-6a4c-4c2d-81ce-932e6b75ad0e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:32.526105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def87db-d4ad-49b5-bf1b-62b08ce95724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:32.526644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def87db-d4ad-49b5-bf1b-62b08ce95724 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:32.526930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def87db-d4ad-49b5-bf1b-62b08ce95724 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:32.527245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def87db-d4ad-49b5-bf1b-62b08ce95724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:32.528873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4def87db-d4ad-49b5-bf1b-62b08ce95724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:32.572700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:32.645277 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def87db-d4ad-49b5-bf1b-62b08ce95724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:32.647442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def87db-d4ad-49b5-bf1b-62b08ce95724 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:32.670310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 676/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:36.647011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115e801-cc1b-44e9-bb78-c0c8a46d7cf2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:36.647760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115e801-cc1b-44e9-bb78-c0c8a46d7cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:36.647760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115e801-cc1b-44e9-bb78-c0c8a46d7cf2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:36.649004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 671/1347] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:36.654352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4cbaa-ead3-4680-a188-a839449eabc7 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:36.654899 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4cbaa-ead3-4680-a188-a839449eabc7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:36.654899 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4cbaa-ead3-4680-a188-a839449eabc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:36.725796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0ec4cbaa-ead3-4680-a188-a839449eabc7 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:36.727186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0ec4cbaa-ead3-4680-a188-a839449eabc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:36.844406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:36.846888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 677/1348] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:36.886498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115e801-cc1b-44e9-bb78-c0c8a46d7cf2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:36.888109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8115e801-cc1b-44e9-bb78-c0c8a46d7cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:36.999888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115e801-cc1b-44e9-bb78-c0c8a46d7cf2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:37.049846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:37.077615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad1f50-34f5-4aba-98bc-a1be17607128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:37.078235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad1f50-34f5-4aba-98bc-a1be17607128 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:37.078519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad1f50-34f5-4aba-98bc-a1be17607128 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:37.078804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad1f50-34f5-4aba-98bc-a1be17607128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:37.080479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0ad1f50-34f5-4aba-98bc-a1be17607128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:37.201965 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad1f50-34f5-4aba-98bc-a1be17607128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:37.203491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad1f50-34f5-4aba-98bc-a1be17607128 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:37.221898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 672/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:42.163002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992a1b4-83db-4209-ae1c-97c84665e76a None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:42.163589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992a1b4-83db-4209-ae1c-97c84665e76a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:42.163904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992a1b4-83db-4209-ae1c-97c84665e76a None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:42.167831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 678/1350] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:29:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:42.176689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302329b5-9db5-43d3-a88e-8ad745a433f3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:42.177685 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302329b5-9db5-43d3-a88e-8ad745a433f3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:42.178773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302329b5-9db5-43d3-a88e-8ad745a433f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:42.293852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-302329b5-9db5-43d3-a88e-8ad745a433f3 None None] MFA Rules not processed for user `928c5d1284b74f1291c031323bb03a20`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:42.295366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-302329b5-9db5-43d3-a88e-8ad745a433f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:42.315969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:42.426680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 673/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:29:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:46.534841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b228e-5b4f-42c0-98fb-a1c4b4f5a756 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:46.535422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b228e-5b4f-42c0-98fb-a1c4b4f5a756 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:46.535422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b228e-5b4f-42c0-98fb-a1c4b4f5a756 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:46.536795 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 679/1352] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:46.542763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a011a-1a94-49c5-a068-c4fbd2d969b6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:46.543233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a011a-1a94-49c5-a068-c4fbd2d969b6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:46.543440 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a011a-1a94-49c5-a068-c4fbd2d969b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:46.624514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4e1a011a-1a94-49c5-a068-c4fbd2d969b6 None None] MFA Rules not processed for user `928c5d1284b74f1291c031323bb03a20`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:46.625809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4e1a011a-1a94-49c5-a068-c4fbd2d969b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:46.699376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:46.738944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 674/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:29:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:48.037374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549622f-8798-408e-a44f-c7106e2ca3ab None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:48.037945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549622f-8798-408e-a44f-c7106e2ca3ab None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:48.037945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549622f-8798-408e-a44f-c7106e2ca3ab None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:48.039309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 680/1354] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:48.044252 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639b46f-65e6-4ff7-8183-f86770ac651a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:48.044693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639b46f-65e6-4ff7-8183-f86770ac651a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:48.044939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639b46f-65e6-4ff7-8183-f86770ac651a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:48.122252 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d639b46f-65e6-4ff7-8183-f86770ac651a None None] MFA Rules not processed for user `928c5d1284b74f1291c031323bb03a20`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:48.123491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d639b46f-65e6-4ff7-8183-f86770ac651a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:48.243151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 675/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:29:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:51.777222 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e9624-4634-4630-bb34-a799a535d9ea None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:51.777762 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e9624-4634-4630-bb34-a799a535d9ea None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:51.777762 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e9624-4634-4630-bb34-a799a535d9ea None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:51.779858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 681/1356] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:51.785126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d546a-6a04-49e6-be1b-5b819b33ca4d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:51.786086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d546a-6a04-49e6-be1b-5b819b33ca4d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:51.786681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d546a-6a04-49e6-be1b-5b819b33ca4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:51.842842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:51.892191 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3c7d546a-6a04-49e6-be1b-5b819b33ca4d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:51.893403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3c7d546a-6a04-49e6-be1b-5b819b33ca4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:52.004911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 676/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:29:52.060140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2972b-d85f-4eed-936e-06eb820d0735 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:52.060896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2972b-d85f-4eed-936e-06eb820d0735 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:52.061532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2972b-d85f-4eed-936e-06eb820d0735 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:52.063302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 682/1358] 10.222.0.40 () {54 vars in 962 bytes} [Mon Aug 1 07:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:52.072047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6d48a-96a8-48f5-a055-81a7e148a9fd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:52.072740 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6d48a-96a8-48f5-a055-81a7e148a9fd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:52.073225 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6d48a-96a8-48f5-a055-81a7e148a9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:52.166697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-01f6d48a-96a8-48f5-a055-81a7e148a9fd None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:52.168302 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-01f6d48a-96a8-48f5-a055-81a7e148a9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:52.222546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:52.305899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 677/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Aug 1 07:29:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:52.313778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2972b-d85f-4eed-936e-06eb820d0735 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:52.315455 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0cb2972b-d85f-4eed-936e-06eb820d0735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:52.468435 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2972b-d85f-4eed-936e-06eb820d0735 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:52.549143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:52.593037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfaa0f-12cf-465b-b4e4-2f476c203b35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:52.593779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfaa0f-12cf-465b-b4e4-2f476c203b35 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:52.594139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfaa0f-12cf-465b-b4e4-2f476c203b35 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:52.594474 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfaa0f-12cf-465b-b4e4-2f476c203b35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:52.596174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-71cfaa0f-12cf-465b-b4e4-2f476c203b35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:52.701275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cfaa0f-12cf-465b-b4e4-2f476c203b35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:52.703613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cfaa0f-12cf-465b-b4e4-2f476c203b35 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:52.726480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 683/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:55.002910 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521754f-1e17-4ed3-abcb-1e9ce686967a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:55.003450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521754f-1e17-4ed3-abcb-1e9ce686967a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:55.003450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521754f-1e17-4ed3-abcb-1e9ce686967a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:55.005158 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 678/1361] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:55.010746 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51eda55-11d5-49f8-a9c3-a9cd6e730735 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:55.011192 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51eda55-11d5-49f8-a9c3-a9cd6e730735 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:55.011420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51eda55-11d5-49f8-a9c3-a9cd6e730735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:55.083784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d51eda55-11d5-49f8-a9c3-a9cd6e730735 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:55.085058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d51eda55-11d5-49f8-a9c3-a9cd6e730735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:55.127614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:55.200365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 684/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:55.251097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521754f-1e17-4ed3-abcb-1e9ce686967a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:55.252971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0521754f-1e17-4ed3-abcb-1e9ce686967a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:55.437459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521754f-1e17-4ed3-abcb-1e9ce686967a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:55.446442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:55.568354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ab037-368f-4695-8cfb-9304d0f31c0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:55.568980 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ab037-368f-4695-8cfb-9304d0f31c0f service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:55.569267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ab037-368f-4695-8cfb-9304d0f31c0f service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:55.569551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ab037-368f-4695-8cfb-9304d0f31c0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:55.571368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-553ab037-368f-4695-8cfb-9304d0f31c0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:55.675644 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ab037-368f-4695-8cfb-9304d0f31c0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:55.677335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ab037-368f-4695-8cfb-9304d0f31c0f service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:55.696640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 679/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:58.062009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c828f0-ec50-4abd-96ce-713bcdaf5954 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:58.063032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c828f0-ec50-4abd-96ce-713bcdaf5954 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:58.063357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c828f0-ec50-4abd-96ce-713bcdaf5954 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:58.065050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 685/1364] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:58.070265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44b739-3af0-4cb6-89ac-680b294154da None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:58.070737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44b739-3af0-4cb6-89ac-680b294154da None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:58.071054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44b739-3af0-4cb6-89ac-680b294154da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:58.095434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:58.156952 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4f44b739-3af0-4cb6-89ac-680b294154da None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:58.158182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f44b739-3af0-4cb6-89ac-680b294154da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:58.262977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 680/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:58.310204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395a561-6cad-454e-94e5-b8de2402e9d6 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:58.310582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395a561-6cad-454e-94e5-b8de2402e9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:58.310939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395a561-6cad-454e-94e5-b8de2402e9d6 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:58.312468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 686/1366] 10.222.0.40 () {54 vars in 962 bytes} [Mon Aug 1 07:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:58.318120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141b120-2e8e-4d1f-ae00-0dd8bc0d55da None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:58.318620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141b120-2e8e-4d1f-ae00-0dd8bc0d55da None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:58.318970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141b120-2e8e-4d1f-ae00-0dd8bc0d55da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:58.397904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4141b120-2e8e-4d1f-ae00-0dd8bc0d55da None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:29:58.399148 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4141b120-2e8e-4d1f-ae00-0dd8bc0d55da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:58.438306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:58.551044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 681/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Aug 1 07:29:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:29:58.561672 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395a561-6cad-454e-94e5-b8de2402e9d6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:29:58.563541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b395a561-6cad-454e-94e5-b8de2402e9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:58.744683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395a561-6cad-454e-94e5-b8de2402e9d6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:29:58.772111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:29:58.892554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0fd24-2527-4e94-83d7-25291a778551 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:29:58.893497 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0fd24-2527-4e94-83d7-25291a778551 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:58.893785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0fd24-2527-4e94-83d7-25291a778551 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:58.894213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0fd24-2527-4e94-83d7-25291a778551 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:58.896841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8cb0fd24-2527-4e94-83d7-25291a778551 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:29:59.020916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0fd24-2527-4e94-83d7-25291a778551 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:29:59.022848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0fd24-2527-4e94-83d7-25291a778551 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:29:59.044923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 687/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:29:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:29:59.972083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed341290-68f8-4cb1-9bfe-5e3a5b79907c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:59.972576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed341290-68f8-4cb1-9bfe-5e3a5b79907c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:59.972660 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed341290-68f8-4cb1-9bfe-5e3a5b79907c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:29:59.976071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 682/1369] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:29:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:29:59.981063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a8052-cc4e-46c2-8684-4dbadea3e83d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:29:59.981539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a8052-cc4e-46c2-8684-4dbadea3e83d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:29:59.981625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a8052-cc4e-46c2-8684-4dbadea3e83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:00.016304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:00.063565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c10a8052-cc4e-46c2-8684-4dbadea3e83d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:00.064791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c10a8052-cc4e-46c2-8684-4dbadea3e83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:00.171126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 688/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:00.522917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8168e8-e072-4f6d-b865-8e0e32ddfcb5 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:00.522917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8168e8-e072-4f6d-b865-8e0e32ddfcb5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:00.523357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8168e8-e072-4f6d-b865-8e0e32ddfcb5 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:00.525665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 683/1371] 10.222.0.40 () {54 vars in 964 bytes} [Mon Aug 1 07:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:00.530846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052314b-1997-427d-b33a-043d2ce28058 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:00.531250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052314b-1997-427d-b33a-043d2ce28058 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:00.531536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052314b-1997-427d-b33a-043d2ce28058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:00.616585 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8052314b-1997-427d-b33a-043d2ce28058 None None] MFA Rules not processed for user `495c489f00d74b53ac02c2cbbb323c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:00.617814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8052314b-1997-427d-b33a-043d2ce28058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:00.669822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:00.701377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 689/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Mon Aug 1 07:30:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:00.709848 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8168e8-e072-4f6d-b865-8e0e32ddfcb5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:00.711388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1d8168e8-e072-4f6d-b865-8e0e32ddfcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:00.819389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8168e8-e072-4f6d-b865-8e0e32ddfcb5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:00.860102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:00.895943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f027eef-661e-4e68-a0f0-31e3c785c864 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:00.896557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f027eef-661e-4e68-a0f0-31e3c785c864 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:00.896840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f027eef-661e-4e68-a0f0-31e3c785c864 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:00.897117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f027eef-661e-4e68-a0f0-31e3c785c864 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:00.898718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5f027eef-661e-4e68-a0f0-31e3c785c864 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:01.025428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f027eef-661e-4e68-a0f0-31e3c785c864 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:01.026891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f027eef-661e-4e68-a0f0-31e3c785c864 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:01.046728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 684/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:01.170174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46165201-0547-48ec-87e1-729453814ae4 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:01.170601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46165201-0547-48ec-87e1-729453814ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:01.170831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46165201-0547-48ec-87e1-729453814ae4 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:01.172136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 690/1374] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:01.178112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0d393-9a88-48b2-9c7f-08243d67ef82 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:01.178709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0d393-9a88-48b2-9c7f-08243d67ef82 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:01.179201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0d393-9a88-48b2-9c7f-08243d67ef82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:01.246152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6ad0d393-9a88-48b2-9c7f-08243d67ef82 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:01.247286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6ad0d393-9a88-48b2-9c7f-08243d67ef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:01.269287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:01.351719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 685/1375] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:30:01.392246 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46165201-0547-48ec-87e1-729453814ae4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:01.393763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-46165201-0547-48ec-87e1-729453814ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:01.543130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46165201-0547-48ec-87e1-729453814ae4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:01.594979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:01.657891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4397229-7cee-4320-a6f4-54bdff46ae53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:01.658515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4397229-7cee-4320-a6f4-54bdff46ae53 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:01.658802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4397229-7cee-4320-a6f4-54bdff46ae53 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:01.659109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4397229-7cee-4320-a6f4-54bdff46ae53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:01.660915 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d4397229-7cee-4320-a6f4-54bdff46ae53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:01.755464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4397229-7cee-4320-a6f4-54bdff46ae53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:01.757112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4397229-7cee-4320-a6f4-54bdff46ae53 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:01.775693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 691/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:03.835086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4f480-b5a8-4a01-8513-557e779f47b2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:03.835621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4f480-b5a8-4a01-8513-557e779f47b2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:03.835716 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4f480-b5a8-4a01-8513-557e779f47b2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:03.837035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 686/1377] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:03.842121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cccaa-4e9c-4848-b422-9918ac901e46 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:03.842526 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cccaa-4e9c-4848-b422-9918ac901e46 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:03.842753 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cccaa-4e9c-4848-b422-9918ac901e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:03.915911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-fd2cccaa-4e9c-4848-b422-9918ac901e46 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:03.917830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-fd2cccaa-4e9c-4848-b422-9918ac901e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:03.925108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:04.055120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 692/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:04.104152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4f480-b5a8-4a01-8513-557e779f47b2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:04.106030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f8c4f480-b5a8-4a01-8513-557e779f47b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:04.252899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4f480-b5a8-4a01-8513-557e779f47b2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:04.257622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:04.374311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10534b2b-4f1f-472e-9af3-d4e7f41e739b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:04.374933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10534b2b-4f1f-472e-9af3-d4e7f41e739b service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:04.375248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10534b2b-4f1f-472e-9af3-d4e7f41e739b service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:04.375526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10534b2b-4f1f-472e-9af3-d4e7f41e739b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:04.377410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-10534b2b-4f1f-472e-9af3-d4e7f41e739b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:04.500668 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10534b2b-4f1f-472e-9af3-d4e7f41e739b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:04.503181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10534b2b-4f1f-472e-9af3-d4e7f41e739b service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:04.538290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 687/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:06.660948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d5a5a-39f3-4cf4-90c5-33dc91bfebda None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:06.661513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d5a5a-39f3-4cf4-90c5-33dc91bfebda None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:06.661791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d5a5a-39f3-4cf4-90c5-33dc91bfebda None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:06.668191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 693/1380] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:30:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:06.669605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2457b5a-f3af-4797-aa8d-7c17c543806d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:06.670110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2457b5a-f3af-4797-aa8d-7c17c543806d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:06.670205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2457b5a-f3af-4797-aa8d-7c17c543806d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:06.683585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:06.743901 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b2457b5a-f3af-4797-aa8d-7c17c543806d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:06.745098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b2457b5a-f3af-4797-aa8d-7c17c543806d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:06.847646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 688/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:06.890784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d5a5a-39f3-4cf4-90c5-33dc91bfebda None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:06.892710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-808d5a5a-39f3-4cf4-90c5-33dc91bfebda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:07.054542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:07.066677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d5a5a-39f3-4cf4-90c5-33dc91bfebda None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:07.184922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53e4dd-867d-483e-8be9-5545fcf66599 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:07.185525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53e4dd-867d-483e-8be9-5545fcf66599 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:07.185801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53e4dd-867d-483e-8be9-5545fcf66599 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:07.186071 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53e4dd-867d-483e-8be9-5545fcf66599 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:07.187685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-aa53e4dd-867d-483e-8be9-5545fcf66599 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:07.281815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53e4dd-867d-483e-8be9-5545fcf66599 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:07.283491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53e4dd-867d-483e-8be9-5545fcf66599 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:07.294845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:07.304723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 694/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:09.596954 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ad95b-a1ea-4869-8781-d172d2686ae2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:09.598497 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ad95b-a1ea-4869-8781-d172d2686ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:09.598497 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ad95b-a1ea-4869-8781-d172d2686ae2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:09.599631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 689/1383] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:30:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:09.605538 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e896bb-631d-470c-9909-a24766b3ae1c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:09.605987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e896bb-631d-470c-9909-a24766b3ae1c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:09.606249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e896bb-631d-470c-9909-a24766b3ae1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:09.682226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-25e896bb-631d-470c-9909-a24766b3ae1c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:09.683497 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-25e896bb-631d-470c-9909-a24766b3ae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:09.789590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 695/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:09.844689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ad95b-a1ea-4869-8781-d172d2686ae2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:09.846555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f4ad95b-a1ea-4869-8781-d172d2686ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:09.980011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:10.095354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ad95b-a1ea-4869-8781-d172d2686ae2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:10.288178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90fbb1-c854-490d-a9d2-fd9ec52d75b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:10.288794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90fbb1-c854-490d-a9d2-fd9ec52d75b6 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:10.289081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90fbb1-c854-490d-a9d2-fd9ec52d75b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:10.293687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90fbb1-c854-490d-a9d2-fd9ec52d75b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:10.293687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ae90fbb1-c854-490d-a9d2-fd9ec52d75b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:10.356823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:10.414941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90fbb1-c854-490d-a9d2-fd9ec52d75b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:10.416729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90fbb1-c854-490d-a9d2-fd9ec52d75b6 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:10.447114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 690/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:12.726562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe3f99-73a9-4a6d-adf6-80f537945af9 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:12.727136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe3f99-73a9-4a6d-adf6-80f537945af9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:12.727256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe3f99-73a9-4a6d-adf6-80f537945af9 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:12.728917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 696/1386] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:30:12.734124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad66468-f2bb-4c05-bc1d-f5e3fd4412f2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:12.734572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad66468-f2bb-4c05-bc1d-f5e3fd4412f2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:12.734711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad66468-f2bb-4c05-bc1d-f5e3fd4412f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:12.802269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0ad66468-f2bb-4c05-bc1d-f5e3fd4412f2 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:12.803431 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0ad66468-f2bb-4c05-bc1d-f5e3fd4412f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:12.901961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 691/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:12.945455 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe3f99-73a9-4a6d-adf6-80f537945af9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:12.947158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4dbe3f99-73a9-4a6d-adf6-80f537945af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:13.031714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:13.109733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe3f99-73a9-4a6d-adf6-80f537945af9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:13.236260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d48f4-42cf-41fd-a785-11cd66e5e9b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:13.236920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d48f4-42cf-41fd-a785-11cd66e5e9b4 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:13.237205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d48f4-42cf-41fd-a785-11cd66e5e9b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:13.237508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d48f4-42cf-41fd-a785-11cd66e5e9b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:13.239869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e15d48f4-42cf-41fd-a785-11cd66e5e9b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:13.281440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:13.337084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d48f4-42cf-41fd-a785-11cd66e5e9b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:13.338777 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d48f4-42cf-41fd-a785-11cd66e5e9b4 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:13.357748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 697/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:15.543833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff053-1026-4e92-bd5a-2dd1a92b185b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:15.544415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff053-1026-4e92-bd5a-2dd1a92b185b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:15.544415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff053-1026-4e92-bd5a-2dd1a92b185b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:15.545925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 692/1389] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:15.551342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a6051-1554-4500-bcfc-1c51e8afb6b8 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:15.551798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a6051-1554-4500-bcfc-1c51e8afb6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:15.551890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a6051-1554-4500-bcfc-1c51e8afb6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:15.621516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-573a6051-1554-4500-bcfc-1c51e8afb6b8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:15.622737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-573a6051-1554-4500-bcfc-1c51e8afb6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:15.734382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 698/1390] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:15.778856 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff053-1026-4e92-bd5a-2dd1a92b185b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:15.780463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-368ff053-1026-4e92-bd5a-2dd1a92b185b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:15.813489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:15.945301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff053-1026-4e92-bd5a-2dd1a92b185b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:16.059465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:16.071344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30946908-41a9-466f-924c-207dd0a49474 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:16.071927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30946908-41a9-466f-924c-207dd0a49474 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:16.072186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30946908-41a9-466f-924c-207dd0a49474 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:16.072475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30946908-41a9-466f-924c-207dd0a49474 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:16.074064 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-30946908-41a9-466f-924c-207dd0a49474 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:16.169105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30946908-41a9-466f-924c-207dd0a49474 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:16.170766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30946908-41a9-466f-924c-207dd0a49474 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:16.193649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 693/1391] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:16.201036 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cb2c7-5a6a-48ff-92d9-31794cf718c0 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:16.201350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cb2c7-5a6a-48ff-92d9-31794cf718c0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:16.201645 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cb2c7-5a6a-48ff-92d9-31794cf718c0 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:16.203024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 699/1392] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:16.208611 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dc91c-6eb4-44f3-a748-4b8995a18850 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:16.208907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dc91c-6eb4-44f3-a748-4b8995a18850 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:16.209189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dc91c-6eb4-44f3-a748-4b8995a18850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:16.287241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ed7dc91c-6eb4-44f3-a748-4b8995a18850 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:16.288616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ed7dc91c-6eb4-44f3-a748-4b8995a18850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:16.367345 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:16.418352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 694/1393] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:16.476662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b91d2-ffce-41c8-b5b2-38367ad584f9 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:16.477205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b91d2-ffce-41c8-b5b2-38367ad584f9 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:16.477785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b91d2-ffce-41c8-b5b2-38367ad584f9 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:16.479277 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 700/1394] 10.222.0.40 () {54 vars in 966 bytes} [Mon Aug 1 07:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:16.484365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482141a0-e782-4749-8f78-66f909f53383 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:16.484826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482141a0-e782-4749-8f78-66f909f53383 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:16.485233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482141a0-e782-4749-8f78-66f909f53383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:16.566249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-482141a0-e782-4749-8f78-66f909f53383 None None] MFA Rules not processed for user `5f57fa108c3d478a9976f4aea5f33bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:16.567497 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-482141a0-e782-4749-8f78-66f909f53383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:16.655105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 695/1395] 10.222.0.40 () {58 vars in 1063 bytes} [Mon Aug 1 07:30:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:16.663477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b91d2-ffce-41c8-b5b2-38367ad584f9 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:16.665148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-584b91d2-ffce-41c8-b5b2-38367ad584f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:16.683486 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:16.778523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b91d2-ffce-41c8-b5b2-38367ad584f9 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:16.852703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b05d1-0416-4935-ac8b-7edc16c2d654 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f57fa108c3d478a9976f4aea5f33bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:16.853317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b05d1-0416-4935-ac8b-7edc16c2d654 service glance] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:16.853596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b05d1-0416-4935-ac8b-7edc16c2d654 service glance] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:16.853871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b05d1-0416-4935-ac8b-7edc16c2d654 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:16.855494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-183b05d1-0416-4935-ac8b-7edc16c2d654 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:16.925347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:16.969028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b05d1-0416-4935-ac8b-7edc16c2d654 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:16.970499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b05d1-0416-4935-ac8b-7edc16c2d654 service glance] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:16.989461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 701/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:17.724341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84431f-61c2-4559-ba67-7afc15078e2b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:17.724828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84431f-61c2-4559-ba67-7afc15078e2b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:17.724929 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84431f-61c2-4559-ba67-7afc15078e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:17.793125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1f84431f-61c2-4559-ba67-7afc15078e2b None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:17.794322 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1f84431f-61c2-4559-ba67-7afc15078e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:17.802554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:17.884767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 696/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:30:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:17.935411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528fb13-e1cf-4920-b1a4-a865e0c7f777 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:17.935872 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528fb13-e1cf-4920-b1a4-a865e0c7f777 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:17.935975 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528fb13-e1cf-4920-b1a4-a865e0c7f777 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:17.937328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 702/1398] 10.222.0.40 () {54 vars in 943 bytes} [Mon Aug 1 07:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:17.943796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d96bfe-2f20-4749-b12f-9a557fd88b01 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:17.944088 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d96bfe-2f20-4749-b12f-9a557fd88b01 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:17.944366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d96bfe-2f20-4749-b12f-9a557fd88b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:18.025025 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-64d96bfe-2f20-4749-b12f-9a557fd88b01 None None] MFA Rules not processed for user `2ba2655c2eef408498c0298d432263f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:18.026273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-64d96bfe-2f20-4749-b12f-9a557fd88b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:18.110882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:18.134688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 697/1399] 10.222.0.40 () {58 vars in 1040 bytes} [Mon Aug 1 07:30:17 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:18.144659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528fb13-e1cf-4920-b1a4-a865e0c7f777 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:18.146338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0528fb13-e1cf-4920-b1a4-a865e0c7f777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:18.283610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528fb13-e1cf-4920-b1a4-a865e0c7f777 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:18.289675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564528a4-e39b-4bb0-ab00-b9c1629368ae None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:18.289965 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564528a4-e39b-4bb0-ab00-b9c1629368ae None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:18.290249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564528a4-e39b-4bb0-ab00-b9c1629368ae None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:18.291701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 698/1400] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:30:18.296759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c016f5-c9d0-4617-9858-1cfd01b59154 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:18.297132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c016f5-c9d0-4617-9858-1cfd01b59154 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:18.297474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c016f5-c9d0-4617-9858-1cfd01b59154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:18.322325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:18.364934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-02c016f5-c9d0-4617-9858-1cfd01b59154 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:18.366147 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-02c016f5-c9d0-4617-9858-1cfd01b59154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:18.392282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a49a42-f458-487c-b3f2-70de61d9e4b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:18.392899 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a49a42-f458-487c-b3f2-70de61d9e4b3 service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:18.393183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a49a42-f458-487c-b3f2-70de61d9e4b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:18.393478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a49a42-f458-487c-b3f2-70de61d9e4b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:18.395725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f5a49a42-f458-487c-b3f2-70de61d9e4b3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:18.467920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 699/1401] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:30:18.478322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a49a42-f458-487c-b3f2-70de61d9e4b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:18.479955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a49a42-f458-487c-b3f2-70de61d9e4b3 service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:18.488670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 703/1402] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:30:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:30:18.513926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c016f5-c9d0-4617-9858-1cfd01b59154 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:18.515167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-02c016f5-c9d0-4617-9858-1cfd01b59154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:18.595198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:18.654660 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c016f5-c9d0-4617-9858-1cfd01b59154 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:18.785681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b7ec7-6e06-401d-b550-5d1963d17adc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:18.786469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b7ec7-6e06-401d-b550-5d1963d17adc service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:18.786865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b7ec7-6e06-401d-b550-5d1963d17adc service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:18.787313 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b7ec7-6e06-401d-b550-5d1963d17adc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:18.789124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-891b7ec7-6e06-401d-b550-5d1963d17adc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:18.852993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:18.888005 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b7ec7-6e06-401d-b550-5d1963d17adc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:18.893258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b7ec7-6e06-401d-b550-5d1963d17adc service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:18.917908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 700/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:21.101700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89087cff-58ac-4bf3-846f-832201447da5 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:21.102233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89087cff-58ac-4bf3-846f-832201447da5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:21.102351 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89087cff-58ac-4bf3-846f-832201447da5 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:21.107104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 704/1404] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:21.109009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace044fc-7628-4761-a37c-b0643890ee34 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:21.109468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace044fc-7628-4761-a37c-b0643890ee34 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:21.109579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace044fc-7628-4761-a37c-b0643890ee34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:21.182978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ace044fc-7628-4761-a37c-b0643890ee34 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:21.184237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ace044fc-7628-4761-a37c-b0643890ee34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:21.296033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 701/1405] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:21.343147 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89087cff-58ac-4bf3-846f-832201447da5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:21.344918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-89087cff-58ac-4bf3-846f-832201447da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:21.434393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:21.523767 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89087cff-58ac-4bf3-846f-832201447da5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:21.656890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f26a48-e489-41df-b2ab-aca9f75ec38f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:21.657794 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f26a48-e489-41df-b2ab-aca9f75ec38f service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:21.658193 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f26a48-e489-41df-b2ab-aca9f75ec38f service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:21.658597 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f26a48-e489-41df-b2ab-aca9f75ec38f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:21.661120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-32f26a48-e489-41df-b2ab-aca9f75ec38f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:21.703020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:21.771783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f26a48-e489-41df-b2ab-aca9f75ec38f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:21.773785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f26a48-e489-41df-b2ab-aca9f75ec38f service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:21.792460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 705/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:23.899302 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d1dc3-076a-489a-98f2-f66fddd2d338 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:23.899827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d1dc3-076a-489a-98f2-f66fddd2d338 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:23.899827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d1dc3-076a-489a-98f2-f66fddd2d338 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:23.901150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 702/1407] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:23.906276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b293c-f253-45cb-bdc2-0b1d955ff893 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:23.906698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b293c-f253-45cb-bdc2-0b1d955ff893 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:23.906904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b293c-f253-45cb-bdc2-0b1d955ff893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:23.995207 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9a7b293c-f253-45cb-bdc2-0b1d955ff893 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:23.996427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9a7b293c-f253-45cb-bdc2-0b1d955ff893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:24.102834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 706/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:30:24.148449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d1dc3-076a-489a-98f2-f66fddd2d338 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:24.150368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f93d1dc3-076a-489a-98f2-f66fddd2d338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:24.199776 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:24.331102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d1dc3-076a-489a-98f2-f66fddd2d338 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:24.453314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:24.460273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d680b-26f1-4b82-a9c8-5a7d9772ba91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:24.461003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d680b-26f1-4b82-a9c8-5a7d9772ba91 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:24.461361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d680b-26f1-4b82-a9c8-5a7d9772ba91 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:24.461735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d680b-26f1-4b82-a9c8-5a7d9772ba91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:24.463483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8a9d680b-26f1-4b82-a9c8-5a7d9772ba91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:24.574697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d680b-26f1-4b82-a9c8-5a7d9772ba91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:24.576632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d680b-26f1-4b82-a9c8-5a7d9772ba91 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:24.597675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 703/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:34.101052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:34.594602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=47 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:40.871838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23146a68-2bc6-45f8-bc54-a1cdd2fd95c6 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:40.872411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23146a68-2bc6-45f8-bc54-a1cdd2fd95c6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:40.872411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23146a68-2bc6-45f8-bc54-a1cdd2fd95c6 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:40.873747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 707/1410] 10.222.0.40 () {54 vars in 918 bytes} [Mon Aug 1 07:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:40.879584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff568fd-50bc-4025-873d-dfec3bf36827 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:40.880041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff568fd-50bc-4025-873d-dfec3bf36827 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:40.880151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff568fd-50bc-4025-873d-dfec3bf36827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:40.956224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3ff568fd-50bc-4025-873d-dfec3bf36827 None None] MFA Rules not processed for user `928c5d1284b74f1291c031323bb03a20`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:40.957438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3ff568fd-50bc-4025-873d-dfec3bf36827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:41.063398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 704/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Mon Aug 1 07:30:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:41.078775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24367309-e3d7-41f9-a18d-e05fda460d34 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:41.079080 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24367309-e3d7-41f9-a18d-e05fda460d34 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:41.079365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24367309-e3d7-41f9-a18d-e05fda460d34 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:41.080934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 708/1412] 10.222.0.40 () {54 vars in 918 bytes} [Mon Aug 1 07:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:41.085570 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa730afd-9f4b-4b04-bbf9-0c6b1f9f6d0c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:41.085854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa730afd-9f4b-4b04-bbf9-0c6b1f9f6d0c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:41.086128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa730afd-9f4b-4b04-bbf9-0c6b1f9f6d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:41.156301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:41.159327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-aa730afd-9f4b-4b04-bbf9-0c6b1f9f6d0c None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:41.161117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-aa730afd-9f4b-4b04-bbf9-0c6b1f9f6d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:41.276780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 705/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Mon Aug 1 07:30:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:30:43.749380 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b73ebc-f085-497f-8fb3-d43410c3a12e None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:43.749907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b73ebc-f085-497f-8fb3-d43410c3a12e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:43.749907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b73ebc-f085-497f-8fb3-d43410c3a12e None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:43.752331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:30:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:43.757981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620252f0-415b-4cb4-8789-aa3cf4d774a3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:43.758739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620252f0-415b-4cb4-8789-aa3cf4d774a3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:43.758739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620252f0-415b-4cb4-8789-aa3cf4d774a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:43.824126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-620252f0-415b-4cb4-8789-aa3cf4d774a3 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:43.825336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-620252f0-415b-4cb4-8789-aa3cf4d774a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:43.879773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:43.921497 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:43.965361 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b73ebc-f085-497f-8fb3-d43410c3a12e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:43.966889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-25b73ebc-f085-497f-8fb3-d43410c3a12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:44.113838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b73ebc-f085-497f-8fb3-d43410c3a12e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:44.185293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:44.227047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029853f-c3d9-46e0-aa61-cf2bfc11c278 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:44.227688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029853f-c3d9-46e0-aa61-cf2bfc11c278 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:44.227973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029853f-c3d9-46e0-aa61-cf2bfc11c278 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:44.228252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029853f-c3d9-46e0-aa61-cf2bfc11c278 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:44.229870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b029853f-c3d9-46e0-aa61-cf2bfc11c278 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:44.319040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029853f-c3d9-46e0-aa61-cf2bfc11c278 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:44.320670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029853f-c3d9-46e0-aa61-cf2bfc11c278 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:44.338734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 710/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:45.234498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afda448-e813-429d-9d39-ba90e9c70bcd None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:45.235032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afda448-e813-429d-9d39-ba90e9c70bcd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:45.235032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afda448-e813-429d-9d39-ba90e9c70bcd None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:45.236750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:45.241549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02f820-e91a-4a6e-83ac-29204402e4c5 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:45.241946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02f820-e91a-4a6e-83ac-29204402e4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:45.242106 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02f820-e91a-4a6e-83ac-29204402e4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:45.317460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-cc02f820-e91a-4a6e-83ac-29204402e4c5 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:45.318668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cc02f820-e91a-4a6e-83ac-29204402e4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:45.345603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:45.415586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:45.567459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afda448-e813-429d-9d39-ba90e9c70bcd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:45.568982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3afda448-e813-429d-9d39-ba90e9c70bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:45.718206 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afda448-e813-429d-9d39-ba90e9c70bcd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:45.758699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:45.830732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e6fc4-1ca8-4d42-b8ac-b3681834a36d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:45.831321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e6fc4-1ca8-4d42-b8ac-b3681834a36d service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:45.831602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e6fc4-1ca8-4d42-b8ac-b3681834a36d service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:45.831876 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e6fc4-1ca8-4d42-b8ac-b3681834a36d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:45.833492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f9e6fc4-1ca8-4d42-b8ac-b3681834a36d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:45.919889 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e6fc4-1ca8-4d42-b8ac-b3681834a36d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:45.921495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e6fc4-1ca8-4d42-b8ac-b3681834a36d service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:45.939762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 708/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:46.822841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0675f0-8ee5-4445-8aa5-6ad05a51f6e1 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:46.823419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0675f0-8ee5-4445-8aa5-6ad05a51f6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:46.823419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0675f0-8ee5-4445-8aa5-6ad05a51f6e1 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:46.824853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:46.829614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e942-5926-4feb-aeda-3a48d169445e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:46.830633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e942-5926-4feb-aeda-3a48d169445e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:46.830968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e942-5926-4feb-aeda-3a48d169445e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:46.884944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:46.907057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f301e942-5926-4feb-aeda-3a48d169445e None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:46.908964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f301e942-5926-4feb-aeda-3a48d169445e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:47.003326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:47.154416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0675f0-8ee5-4445-8aa5-6ad05a51f6e1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:47.155893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5b0675f0-8ee5-4445-8aa5-6ad05a51f6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:47.421911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0675f0-8ee5-4445-8aa5-6ad05a51f6e1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:47.434475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:47.534927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca878c-b78e-41e3-a86a-03540e627992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:47.535572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca878c-b78e-41e3-a86a-03540e627992 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:47.535934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca878c-b78e-41e3-a86a-03540e627992 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:47.536218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca878c-b78e-41e3-a86a-03540e627992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:47.537829 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-60ca878c-b78e-41e3-a86a-03540e627992 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:47.624724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca878c-b78e-41e3-a86a-03540e627992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:47.626609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca878c-b78e-41e3-a86a-03540e627992 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:47.645155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 713/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:48.503400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8bac1-5760-4830-a952-1201c73c090a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:48.503968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8bac1-5760-4830-a952-1201c73c090a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:48.503968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8bac1-5760-4830-a952-1201c73c090a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:48.505341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:30:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:48.510303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d52b-30dd-435a-9e0a-b6c81a8e76ac None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:48.510690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d52b-30dd-435a-9e0a-b6c81a8e76ac None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:48.510848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d52b-30dd-435a-9e0a-b6c81a8e76ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:48.530153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:48.585290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9ff8d52b-30dd-435a-9e0a-b6c81a8e76ac None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:48.586491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9ff8d52b-30dd-435a-9e0a-b6c81a8e76ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:48.680788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:30:48.834463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8bac1-5760-4830-a952-1201c73c090a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:48.835943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a1d8bac1-5760-4830-a952-1201c73c090a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:48.946336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:48.984722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8bac1-5760-4830-a952-1201c73c090a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:49.217793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d78918-cdef-4f06-a659-74b72b8a1833 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:49.218407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d78918-cdef-4f06-a659-74b72b8a1833 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:49.218686 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d78918-cdef-4f06-a659-74b72b8a1833 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:49.218958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d78918-cdef-4f06-a659-74b72b8a1833 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:49.221334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d7d78918-cdef-4f06-a659-74b72b8a1833 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:49.294761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:49.308526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d78918-cdef-4f06-a659-74b72b8a1833 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:49.310112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d78918-cdef-4f06-a659-74b72b8a1833 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:49.327734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 711/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:50.196615 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913b275-a645-44fc-b42d-6f4f9fade305 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:50.197149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913b275-a645-44fc-b42d-6f4f9fade305 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:50.197149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913b275-a645-44fc-b42d-6f4f9fade305 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:50.198501 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:50.203342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e480c4-08b7-4043-a73b-fd9d922b08a9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:50.203773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e480c4-08b7-4043-a73b-fd9d922b08a9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:50.203933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e480c4-08b7-4043-a73b-fd9d922b08a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:50.278561 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c3e480c4-08b7-4043-a73b-fd9d922b08a9 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:50.279722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c3e480c4-08b7-4043-a73b-fd9d922b08a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:50.373911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:50.524349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913b275-a645-44fc-b42d-6f4f9fade305 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:50.525837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b913b275-a645-44fc-b42d-6f4f9fade305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:50.610141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:50.672894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913b275-a645-44fc-b42d-6f4f9fade305 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:50.783745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ac532-a376-40f2-b79c-a984d567bdb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:50.784516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ac532-a376-40f2-b79c-a984d567bdb2 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:50.784805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ac532-a376-40f2-b79c-a984d567bdb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:50.785084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ac532-a376-40f2-b79c-a984d567bdb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:50.786692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-fc6ac532-a376-40f2-b79c-a984d567bdb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:50.838566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:50.880416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ac532-a376-40f2-b79c-a984d567bdb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:50.881933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ac532-a376-40f2-b79c-a984d567bdb2 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:50.900163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 716/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:51.706207 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2329132-418e-47d4-a508-b0e850a25c41 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:51.706625 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2329132-418e-47d4-a508-b0e850a25c41 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:51.706764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2329132-418e-47d4-a508-b0e850a25c41 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:51.708173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:30:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:51.713225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3feb93-1687-40be-b6ac-f2b7a28fba06 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:51.713627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3feb93-1687-40be-b6ac-f2b7a28fba06 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:51.713816 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3feb93-1687-40be-b6ac-f2b7a28fba06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:51.789821 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8a3feb93-1687-40be-b6ac-f2b7a28fba06 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:51.791040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8a3feb93-1687-40be-b6ac-f2b7a28fba06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:51.887594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:52.041794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2329132-418e-47d4-a508-b0e850a25c41 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:52.043349 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b2329132-418e-47d4-a508-b0e850a25c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:52.092463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:52.188454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2329132-418e-47d4-a508-b0e850a25c41 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:52.298168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdfbab-a758-43a0-9cd6-8f4601567fcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:52.299914 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdfbab-a758-43a0-9cd6-8f4601567fcd service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:52.300209 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdfbab-a758-43a0-9cd6-8f4601567fcd service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:52.300490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdfbab-a758-43a0-9cd6-8f4601567fcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:52.302143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-31fdfbab-a758-43a0-9cd6-8f4601567fcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:52.309376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:52.392370 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdfbab-a758-43a0-9cd6-8f4601567fcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:52.393990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdfbab-a758-43a0-9cd6-8f4601567fcd service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:52.411724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 714/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:53.204927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7708214-ef4b-42ef-afb5-98e1bf479786 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:53.205423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7708214-ef4b-42ef-afb5-98e1bf479786 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:53.205554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7708214-ef4b-42ef-afb5-98e1bf479786 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:53.206966 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:53.212223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17644f16-39ea-4fc5-9609-77190abed629 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:53.212637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17644f16-39ea-4fc5-9609-77190abed629 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:53.212775 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17644f16-39ea-4fc5-9609-77190abed629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:53.287034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-17644f16-39ea-4fc5-9609-77190abed629 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:53.288181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-17644f16-39ea-4fc5-9609-77190abed629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:53.355228 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:53.384480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:30:53.533985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7708214-ef4b-42ef-afb5-98e1bf479786 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:53.535592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b7708214-ef4b-42ef-afb5-98e1bf479786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:53.554364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:53.683738 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7708214-ef4b-42ef-afb5-98e1bf479786 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:53.771696 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:53.797394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfff5a-480f-4b35-bf75-a8d67514f6be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:53.798013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfff5a-480f-4b35-bf75-a8d67514f6be service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:53.798300 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfff5a-480f-4b35-bf75-a8d67514f6be service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:53.799176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfff5a-480f-4b35-bf75-a8d67514f6be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:53.801004 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-90cfff5a-480f-4b35-bf75-a8d67514f6be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:53.890068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfff5a-480f-4b35-bf75-a8d67514f6be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:53.891614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfff5a-480f-4b35-bf75-a8d67514f6be service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:53.909510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 719/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:54.722339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4451dc5-de9c-494e-8983-429b73584dc3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:54.722866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4451dc5-de9c-494e-8983-429b73584dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:54.722866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4451dc5-de9c-494e-8983-429b73584dc3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:54.724222 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:30:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:54.729139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f841d-cd60-4632-87fe-9a017d03539d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:54.729549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f841d-cd60-4632-87fe-9a017d03539d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:54.729707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f841d-cd60-4632-87fe-9a017d03539d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:54.806466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bd5f841d-cd60-4632-87fe-9a017d03539d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:54.807726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bd5f841d-cd60-4632-87fe-9a017d03539d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:54.848978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:54.904329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:55.057772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4451dc5-de9c-494e-8983-429b73584dc3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:55.059250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c4451dc5-de9c-494e-8983-429b73584dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:55.203309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4451dc5-de9c-494e-8983-429b73584dc3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:55.257286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:55.319416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af5ed8-23a3-4267-b422-a59c3b0d39d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:55.319944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af5ed8-23a3-4267-b422-a59c3b0d39d2 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:55.320227 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af5ed8-23a3-4267-b422-a59c3b0d39d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:55.320503 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af5ed8-23a3-4267-b422-a59c3b0d39d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:55.322109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e8af5ed8-23a3-4267-b422-a59c3b0d39d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:55.410392 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af5ed8-23a3-4267-b422-a59c3b0d39d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:55.411992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af5ed8-23a3-4267-b422-a59c3b0d39d2 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:55.429692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 717/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:56.241308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d5642-f97e-42ff-8429-fade5498c43d None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:56.241841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d5642-f97e-42ff-8429-fade5498c43d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:56.241841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d5642-f97e-42ff-8429-fade5498c43d None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:56.243214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:56.248080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f2cb8-2d81-415e-a9b2-b1921e4effa3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:56.248466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f2cb8-2d81-415e-a9b2-b1921e4effa3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:56.248641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f2cb8-2d81-415e-a9b2-b1921e4effa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:56.324592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fb7f2cb8-2d81-415e-a9b2-b1921e4effa3 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:56.325801 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fb7f2cb8-2d81-415e-a9b2-b1921e4effa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:56.327851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:56.423441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:56.576958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d5642-f97e-42ff-8429-fade5498c43d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:56.578478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-721d5642-f97e-42ff-8429-fade5498c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:56.728800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d5642-f97e-42ff-8429-fade5498c43d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:56.756759 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:56.840535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1007f-b921-41bc-abcc-2e45570900f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:56.841074 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1007f-b921-41bc-abcc-2e45570900f4 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:56.841350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1007f-b921-41bc-abcc-2e45570900f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:56.841625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1007f-b921-41bc-abcc-2e45570900f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:56.843244 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9ef1007f-b921-41bc-abcc-2e45570900f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:56.931619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1007f-b921-41bc-abcc-2e45570900f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:56.933206 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1007f-b921-41bc-abcc-2e45570900f4 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:56.951995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 722/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:30:58.856359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1978e3a-1698-4107-80a2-63fa0b57d343 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:58.856881 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1978e3a-1698-4107-80a2-63fa0b57d343 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:58.856881 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1978e3a-1698-4107-80a2-63fa0b57d343 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:58.858194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:30:58.863668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542a895-aa19-46bb-aa00-853b23b24857 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:58.864074 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542a895-aa19-46bb-aa00-853b23b24857 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:58.864223 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542a895-aa19-46bb-aa00-853b23b24857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:58.905805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:58.930957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e542a895-aa19-46bb-aa00-853b23b24857 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:30:58.932197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e542a895-aa19-46bb-aa00-853b23b24857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:59.030768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:30:59.070030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1978e3a-1698-4107-80a2-63fa0b57d343 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:30:59.071592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c1978e3a-1698-4107-80a2-63fa0b57d343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:59.179604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1978e3a-1698-4107-80a2-63fa0b57d343 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:30:59.214422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:30:59.258020 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a4423-47bc-42a9-b7d8-06ab3c9c362a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:30:59.258640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a4423-47bc-42a9-b7d8-06ab3c9c362a service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:30:59.258922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a4423-47bc-42a9-b7d8-06ab3c9c362a service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:30:59.259226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a4423-47bc-42a9-b7d8-06ab3c9c362a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:30:59.260854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e45a4423-47bc-42a9-b7d8-06ab3c9c362a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:30:59.370890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a4423-47bc-42a9-b7d8-06ab3c9c362a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:30:59.372365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a4423-47bc-42a9-b7d8-06ab3c9c362a service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:30:59.390838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 720/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:30:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:01.556651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985de5a8-f7d1-46f3-866c-43693ba5a4c1 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:01.557257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985de5a8-f7d1-46f3-866c-43693ba5a4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:01.557411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985de5a8-f7d1-46f3-866c-43693ba5a4c1 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:01.558803 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:01.564050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc24941-4849-44b9-aeaa-44e405e838e8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:01.564823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc24941-4849-44b9-aeaa-44e405e838e8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:01.564823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc24941-4849-44b9-aeaa-44e405e838e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:01.630211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-abc24941-4849-44b9-aeaa-44e405e838e8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:01.631450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-abc24941-4849-44b9-aeaa-44e405e838e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:01.644099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:01.729555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:01.767490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985de5a8-f7d1-46f3-866c-43693ba5a4c1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:01.769149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-985de5a8-f7d1-46f3-866c-43693ba5a4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:01.873493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985de5a8-f7d1-46f3-866c-43693ba5a4c1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:01.885674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:01.947331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940a4de-e484-4f01-9faa-4de66004577a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:01.947954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940a4de-e484-4f01-9faa-4de66004577a service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:01.948240 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940a4de-e484-4f01-9faa-4de66004577a service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:01.948521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940a4de-e484-4f01-9faa-4de66004577a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:01.950141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9940a4de-e484-4f01-9faa-4de66004577a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:02.059817 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940a4de-e484-4f01-9faa-4de66004577a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:02.061269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940a4de-e484-4f01-9faa-4de66004577a service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:02.080542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 725/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:04.394590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4abdc9-dbe2-4827-9757-eefbfa1f9ff5 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:04.395144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4abdc9-dbe2-4827-9757-eefbfa1f9ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:04.395144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4abdc9-dbe2-4827-9757-eefbfa1f9ff5 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:04.396455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:04.401633 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582221a-faa5-4c5b-91f7-fb40a5a5f3b6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:04.402041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582221a-faa5-4c5b-91f7-fb40a5a5f3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:04.402186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582221a-faa5-4c5b-91f7-fb40a5a5f3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:04.449737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:04.466865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5582221a-faa5-4c5b-91f7-fb40a5a5f3b6 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:04.468094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5582221a-faa5-4c5b-91f7-fb40a5a5f3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:04.563482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:04.604407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4abdc9-dbe2-4827-9757-eefbfa1f9ff5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:04.605935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9c4abdc9-dbe2-4827-9757-eefbfa1f9ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:04.708720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4abdc9-dbe2-4827-9757-eefbfa1f9ff5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:04.750520 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:04.785384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76e49a-9c0e-4560-974b-8a3e2f49402b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:04.785997 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76e49a-9c0e-4560-974b-8a3e2f49402b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:04.786280 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76e49a-9c0e-4560-974b-8a3e2f49402b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:04.786564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76e49a-9c0e-4560-974b-8a3e2f49402b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:04.788217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2c76e49a-9c0e-4560-974b-8a3e2f49402b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:04.894700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76e49a-9c0e-4560-974b-8a3e2f49402b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:04.896150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76e49a-9c0e-4560-974b-8a3e2f49402b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:04.913768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 723/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:07.053336 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c50c2b-56c6-4448-84cc-283cc323cae2 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:07.053879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c50c2b-56c6-4448-84cc-283cc323cae2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:07.053879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c50c2b-56c6-4448-84cc-283cc323cae2 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:07.055334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:07.060474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac66f48-10ef-4fc7-a617-52ee87ea9e4a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:07.060865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac66f48-10ef-4fc7-a617-52ee87ea9e4a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:07.061041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac66f48-10ef-4fc7-a617-52ee87ea9e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:07.127737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0ac66f48-10ef-4fc7-a617-52ee87ea9e4a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:07.128953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0ac66f48-10ef-4fc7-a617-52ee87ea9e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:07.149675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:07.225730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:07.265533 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c50c2b-56c6-4448-84cc-283cc323cae2 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:07.267040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-83c50c2b-56c6-4448-84cc-283cc323cae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:07.369426 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c50c2b-56c6-4448-84cc-283cc323cae2 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:07.389454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:07.442567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a048d-2014-4e0c-b6e3-350cf516e9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:07.443203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a048d-2014-4e0c-b6e3-350cf516e9da service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:07.443484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a048d-2014-4e0c-b6e3-350cf516e9da service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:07.443756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a048d-2014-4e0c-b6e3-350cf516e9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:07.445340 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3a5a048d-2014-4e0c-b6e3-350cf516e9da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:07.554970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a048d-2014-4e0c-b6e3-350cf516e9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:07.556359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a048d-2014-4e0c-b6e3-350cf516e9da service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:07.575342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 728/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:09.712878 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9cbd3-c1d9-4a67-b992-683c8449402e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:09.713286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9cbd3-c1d9-4a67-b992-683c8449402e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:09.713437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9cbd3-c1d9-4a67-b992-683c8449402e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:09.714815 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:09.720013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002813ec-87b0-4335-aef4-525821be59e1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:09.720441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002813ec-87b0-4335-aef4-525821be59e1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:09.720626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002813ec-87b0-4335-aef4-525821be59e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:09.783537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:09.786421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-002813ec-87b0-4335-aef4-525821be59e1 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:09.788235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-002813ec-87b0-4335-aef4-525821be59e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:09.882795 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:09.920843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9cbd3-c1d9-4a67-b992-683c8449402e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:09.922397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4ee9cbd3-c1d9-4a67-b992-683c8449402e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:10.025194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9cbd3-c1d9-4a67-b992-683c8449402e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:10.073453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:10.098242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a529089-8165-4ed8-9362-c9339a1c574f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:10.098860 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a529089-8165-4ed8-9362-c9339a1c574f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:10.099174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a529089-8165-4ed8-9362-c9339a1c574f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:10.099455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a529089-8165-4ed8-9362-c9339a1c574f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:10.101086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9a529089-8165-4ed8-9362-c9339a1c574f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:10.209103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a529089-8165-4ed8-9362-c9339a1c574f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:10.210569 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a529089-8165-4ed8-9362-c9339a1c574f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:10.228961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 726/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:12.248161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17d564-6e06-4e5a-9b64-5abd779d215c None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:12.248654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17d564-6e06-4e5a-9b64-5abd779d215c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:12.248654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17d564-6e06-4e5a-9b64-5abd779d215c None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:12.249988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:12.256120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2063afa-61c2-4907-8553-3f5be9cb2a66 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:12.256531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2063afa-61c2-4907-8553-3f5be9cb2a66 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:12.256731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2063afa-61c2-4907-8553-3f5be9cb2a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:12.323335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e2063afa-61c2-4907-8553-3f5be9cb2a66 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:12.324571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e2063afa-61c2-4907-8553-3f5be9cb2a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:12.354111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:12.422188 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:31:12.460346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17d564-6e06-4e5a-9b64-5abd779d215c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:12.461843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7b17d564-6e06-4e5a-9b64-5abd779d215c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:12.568150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17d564-6e06-4e5a-9b64-5abd779d215c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:12.596070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:12.643365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2125c5-1097-49e9-8349-3a48538ec4c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:12.643987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2125c5-1097-49e9-8349-3a48538ec4c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:12.644271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2125c5-1097-49e9-8349-3a48538ec4c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:12.644549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2125c5-1097-49e9-8349-3a48538ec4c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:12.646167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ba2125c5-1097-49e9-8349-3a48538ec4c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:12.764012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2125c5-1097-49e9-8349-3a48538ec4c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:12.765466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2125c5-1097-49e9-8349-3a48538ec4c6 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:12.789570 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 731/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:14.937142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34dec9-2d42-44d6-8f88-07c7f92d971f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:14.937720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34dec9-2d42-44d6-8f88-07c7f92d971f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:14.937720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34dec9-2d42-44d6-8f88-07c7f92d971f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:14.939052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:14.944741 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1807a-4f75-409e-b9b7-56820c852d9f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:14.945209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1807a-4f75-409e-b9b7-56820c852d9f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:14.945427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1807a-4f75-409e-b9b7-56820c852d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:15.013182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d4b1807a-4f75-409e-b9b7-56820c852d9f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:15.014391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d4b1807a-4f75-409e-b9b7-56820c852d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:15.023642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:15.114374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:15.152317 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34dec9-2d42-44d6-8f88-07c7f92d971f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:15.153874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ea34dec9-2d42-44d6-8f88-07c7f92d971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:15.255789 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34dec9-2d42-44d6-8f88-07c7f92d971f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:15.310543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:15.329093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b38c0a-87ce-492b-bb87-b775984d55a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:15.329736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38c0a-87ce-492b-bb87-b775984d55a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:15.330570 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38c0a-87ce-492b-bb87-b775984d55a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:15.330846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38c0a-87ce-492b-bb87-b775984d55a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:15.332506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d8b38c0a-87ce-492b-bb87-b775984d55a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:15.441037 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b38c0a-87ce-492b-bb87-b775984d55a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:15.442465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b38c0a-87ce-492b-bb87-b775984d55a4 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:15.460022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 729/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:17.521713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0b0fb-7597-41c1-a79f-46242ffbfb1e None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:17.522278 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0b0fb-7597-41c1-a79f-46242ffbfb1e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:17.522278 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0b0fb-7597-41c1-a79f-46242ffbfb1e None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:17.523696 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:17.528818 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a00312-5e38-456d-b915-6f22851aa451 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:17.529245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a00312-5e38-456d-b915-6f22851aa451 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:17.529381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a00312-5e38-456d-b915-6f22851aa451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:17.596336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-62a00312-5e38-456d-b915-6f22851aa451 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:17.597542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-62a00312-5e38-456d-b915-6f22851aa451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:17.635284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:17.693713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:17.731686 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0b0fb-7597-41c1-a79f-46242ffbfb1e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:17.733223 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-52e0b0fb-7597-41c1-a79f-46242ffbfb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:17.838325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0b0fb-7597-41c1-a79f-46242ffbfb1e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:17.878073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:17.913298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39100cc-72b9-44fe-ac87-55c5caa1e767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:17.913918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39100cc-72b9-44fe-ac87-55c5caa1e767 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:17.914198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39100cc-72b9-44fe-ac87-55c5caa1e767 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:17.914476 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39100cc-72b9-44fe-ac87-55c5caa1e767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:17.916138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c39100cc-72b9-44fe-ac87-55c5caa1e767 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:18.033848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39100cc-72b9-44fe-ac87-55c5caa1e767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:18.035313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39100cc-72b9-44fe-ac87-55c5caa1e767 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:18.053732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 734/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:20.242250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d073e84-81a7-4747-a471-fc4e7cad7552 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:20.242811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d073e84-81a7-4747-a471-fc4e7cad7552 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:20.242811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d073e84-81a7-4747-a471-fc4e7cad7552 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:20.244196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:20.249367 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744ed55-8757-4c4d-8eee-28fdda1b4aba None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:20.249796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744ed55-8757-4c4d-8eee-28fdda1b4aba None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:20.249987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744ed55-8757-4c4d-8eee-28fdda1b4aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:20.316404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e744ed55-8757-4c4d-8eee-28fdda1b4aba None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:20.317630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e744ed55-8757-4c4d-8eee-28fdda1b4aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:20.335263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:20.412005 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:20.448350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d073e84-81a7-4747-a471-fc4e7cad7552 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:20.449904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6d073e84-81a7-4747-a471-fc4e7cad7552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:20.550767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d073e84-81a7-4747-a471-fc4e7cad7552 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:20.616363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:20.623535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db8838-20f7-41ae-814c-8c050a131a50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:20.624156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db8838-20f7-41ae-814c-8c050a131a50 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:20.624444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db8838-20f7-41ae-814c-8c050a131a50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:20.624723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db8838-20f7-41ae-814c-8c050a131a50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:20.626348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c7db8838-20f7-41ae-814c-8c050a131a50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:20.737062 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db8838-20f7-41ae-814c-8c050a131a50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:20.738504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db8838-20f7-41ae-814c-8c050a131a50 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:20.756454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 732/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:22.737905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ea162-6f98-482e-91be-fce3dab6a45b None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:22.738475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ea162-6f98-482e-91be-fce3dab6a45b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:22.738475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ea162-6f98-482e-91be-fce3dab6a45b None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:22.739834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:22.745102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0188f-36f0-4c60-beea-1c85bd68d432 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:22.745485 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0188f-36f0-4c60-beea-1c85bd68d432 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:22.745661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0188f-36f0-4c60-beea-1c85bd68d432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:22.811253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7fd0188f-36f0-4c60-beea-1c85bd68d432 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:22.812457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7fd0188f-36f0-4c60-beea-1c85bd68d432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:22.861041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:22.909154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:31:22.945849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ea162-6f98-482e-91be-fce3dab6a45b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:22.947422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-453ea162-6f98-482e-91be-fce3dab6a45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:23.052032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ea162-6f98-482e-91be-fce3dab6a45b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:23.098138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:23.123937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d5664-13d3-4ff0-bf2e-340423ab0128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:23.124554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d5664-13d3-4ff0-bf2e-340423ab0128 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:23.124835 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d5664-13d3-4ff0-bf2e-340423ab0128 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:23.125109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d5664-13d3-4ff0-bf2e-340423ab0128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:23.127678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-324d5664-13d3-4ff0-bf2e-340423ab0128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:23.233779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d5664-13d3-4ff0-bf2e-340423ab0128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:23.235241 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d5664-13d3-4ff0-bf2e-340423ab0128 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:23.253903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 737/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:32.907342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=22 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:33.250802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=66 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:42.730985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb5e4d-81b8-4113-868f-13d3db45a8ef None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:42.731625 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb5e4d-81b8-4113-868f-13d3db45a8ef None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:42.731625 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb5e4d-81b8-4113-868f-13d3db45a8ef None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:42.732927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 734/1471] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:42.738747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac5d58-4282-4813-a17d-2351510f3d09 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:42.739167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac5d58-4282-4813-a17d-2351510f3d09 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:42.739326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac5d58-4282-4813-a17d-2351510f3d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:42.804843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0dac5d58-4282-4813-a17d-2351510f3d09 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:42.806042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0dac5d58-4282-4813-a17d-2351510f3d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:42.900065 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 738/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:42.985475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d2503-7aab-42c1-996f-88cb71e5bb31 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:42.985701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d2503-7aab-42c1-996f-88cb71e5bb31 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:42.986727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d2503-7aab-42c1-996f-88cb71e5bb31 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:42.988251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 735/1473] 10.222.0.40 () {54 vars in 966 bytes} [Mon Aug 1 07:31:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:42.992921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a751dac-3298-424d-84ea-9116a2cf0015 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:42.993206 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a751dac-3298-424d-84ea-9116a2cf0015 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:42.993482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a751dac-3298-424d-84ea-9116a2cf0015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:43.068190 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4a751dac-3298-424d-84ea-9116a2cf0015 None None] MFA Rules not processed for user `5f57fa108c3d478a9976f4aea5f33bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:43.069408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4a751dac-3298-424d-84ea-9116a2cf0015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:43.089976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:43.145021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 739/1474] 10.222.0.40 () {58 vars in 1063 bytes} [Mon Aug 1 07:31:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:43.152420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d2503-7aab-42c1-996f-88cb71e5bb31 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:43.153926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c50d2503-7aab-42c1-996f-88cb71e5bb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:43.255026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d2503-7aab-42c1-996f-88cb71e5bb31 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:43.327609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bebeb-2204-4b7a-9414-30766e78c70e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f57fa108c3d478a9976f4aea5f33bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:43.328253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bebeb-2204-4b7a-9414-30766e78c70e service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:43.328538 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bebeb-2204-4b7a-9414-30766e78c70e service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:43.328824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bebeb-2204-4b7a-9414-30766e78c70e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:43.330654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f23bebeb-2204-4b7a-9414-30766e78c70e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:43.384216 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:43.437212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bebeb-2204-4b7a-9414-30766e78c70e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:43.438651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bebeb-2204-4b7a-9414-30766e78c70e service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:43.457510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 736/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:45.573054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee462c-b4fa-4428-a281-4c312d892fbb None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:45.573625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee462c-b4fa-4428-a281-4c312d892fbb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:45.573625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee462c-b4fa-4428-a281-4c312d892fbb None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:45.575052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 740/1476] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:45.580480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77437ed8-604d-474a-9064-b2d8cb514f80 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:45.580961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77437ed8-604d-474a-9064-b2d8cb514f80 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:45.581054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77437ed8-604d-474a-9064-b2d8cb514f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:45.647453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-77437ed8-604d-474a-9064-b2d8cb514f80 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:45.648662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-77437ed8-604d-474a-9064-b2d8cb514f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:45.745172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 737/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:45.768941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee462c-b4fa-4428-a281-4c312d892fbb None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:45.770502 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1cee462c-b4fa-4428-a281-4c312d892fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:45.895503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee462c-b4fa-4428-a281-4c312d892fbb None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:45.943810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:45.990328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2554a-05b8-4b9f-83bd-09d88a287357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:45.990947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2554a-05b8-4b9f-83bd-09d88a287357 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:45.991269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2554a-05b8-4b9f-83bd-09d88a287357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:45.991543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2554a-05b8-4b9f-83bd-09d88a287357 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:45.992235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2554a-05b8-4b9f-83bd-09d88a287357 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:45.993969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-aba2554a-05b8-4b9f-83bd-09d88a287357 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:31:45.993969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 01 07:31:45.993969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: Aug 01 07:31:45.995142 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2554a-05b8-4b9f-83bd-09d88a287357 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:46.013884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 741/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Aug 1 07:31:45 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:31:48.016953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd4da2-5bb3-478e-b4af-d84202b96dee None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:48.017555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd4da2-5bb3-478e-b4af-d84202b96dee None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:48.017933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd4da2-5bb3-478e-b4af-d84202b96dee None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:48.019348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 738/1479] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:31:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:31:48.024378 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f3a65-9353-4c99-8219-590ae96d727d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:48.024812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f3a65-9353-4c99-8219-590ae96d727d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:48.024916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f3a65-9353-4c99-8219-590ae96d727d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:48.089169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c31f3a65-9353-4c99-8219-590ae96d727d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:48.090363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c31f3a65-9353-4c99-8219-590ae96d727d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:48.187835 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 742/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:48.230355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd4da2-5bb3-478e-b4af-d84202b96dee None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:48.232198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-29bd4da2-5bb3-478e-b4af-d84202b96dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:48.243074 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:48.376868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd4da2-5bb3-478e-b4af-d84202b96dee None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:48.455234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:48.487729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6735af-49ec-49ba-b9ee-43618243f119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:48.488340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6735af-49ec-49ba-b9ee-43618243f119 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:48.488620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6735af-49ec-49ba-b9ee-43618243f119 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:48.488892 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6735af-49ec-49ba-b9ee-43618243f119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:48.490514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9e6735af-49ec-49ba-b9ee-43618243f119 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:48.577179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6735af-49ec-49ba-b9ee-43618243f119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:48.578761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6735af-49ec-49ba-b9ee-43618243f119 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:48.597552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 739/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:50.764484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956812a9-8afb-49bf-9ee5-62f70b380aac None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:50.765091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956812a9-8afb-49bf-9ee5-62f70b380aac None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:50.765091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956812a9-8afb-49bf-9ee5-62f70b380aac None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:50.766432 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 743/1482] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:50.771864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8337f-7778-444b-b142-aab4c318e8b7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:50.772307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8337f-7778-444b-b142-aab4c318e8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:50.772497 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8337f-7778-444b-b142-aab4c318e8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:50.837853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-66c8337f-7778-444b-b142-aab4c318e8b7 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:50.839056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-66c8337f-7778-444b-b142-aab4c318e8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:50.884794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:50.935121 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 740/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:50.973067 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956812a9-8afb-49bf-9ee5-62f70b380aac None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:50.974603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-956812a9-8afb-49bf-9ee5-62f70b380aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:50.998500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:51.078087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956812a9-8afb-49bf-9ee5-62f70b380aac None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:51.150933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1280e0-f101-4ddb-bef4-2add435f8a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:51.151584 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1280e0-f101-4ddb-bef4-2add435f8a4c service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:51.151869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1280e0-f101-4ddb-bef4-2add435f8a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:51.152145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1280e0-f101-4ddb-bef4-2add435f8a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:51.153744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9e1280e0-f101-4ddb-bef4-2add435f8a4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:51.227957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:51.263615 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1280e0-f101-4ddb-bef4-2add435f8a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:51.265053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1280e0-f101-4ddb-bef4-2add435f8a4c service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:51.283922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 744/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:53.322014 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927d203-8b8e-42d5-ae8d-1e0c55933ac3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:53.322525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927d203-8b8e-42d5-ae8d-1e0c55933ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:53.322525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927d203-8b8e-42d5-ae8d-1e0c55933ac3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:53.323866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 741/1485] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:53.329013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4a699-5746-40c5-ac5c-2f2c998ee3ef None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:53.329463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4a699-5746-40c5-ac5c-2f2c998ee3ef None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:53.329684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4a699-5746-40c5-ac5c-2f2c998ee3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:53.394396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3cc4a699-5746-40c5-ac5c-2f2c998ee3ef None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:53.395635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3cc4a699-5746-40c5-ac5c-2f2c998ee3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:53.491821 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 745/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:53.530251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927d203-8b8e-42d5-ae8d-1e0c55933ac3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:53.531844 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5927d203-8b8e-42d5-ae8d-1e0c55933ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:53.637526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927d203-8b8e-42d5-ae8d-1e0c55933ac3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:53.711146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:53.713198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31012f6c-6450-4e96-80b6-5a65abafdda0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:53.713855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31012f6c-6450-4e96-80b6-5a65abafdda0 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:53.714339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31012f6c-6450-4e96-80b6-5a65abafdda0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:53.714662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31012f6c-6450-4e96-80b6-5a65abafdda0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:53.716727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-31012f6c-6450-4e96-80b6-5a65abafdda0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:53.826153 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31012f6c-6450-4e96-80b6-5a65abafdda0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:53.827608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31012f6c-6450-4e96-80b6-5a65abafdda0 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:53.846096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 742/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:55.937865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2667f-5ec4-4b46-9404-103d3e44d6d2 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:55.938448 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2667f-5ec4-4b46-9404-103d3e44d6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:55.938448 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2667f-5ec4-4b46-9404-103d3e44d6d2 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:55.940015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 746/1488] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:31:55.945299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305eae2d-a6ca-4767-880c-8e60882ca761 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:55.945711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305eae2d-a6ca-4767-880c-8e60882ca761 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:55.945859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305eae2d-a6ca-4767-880c-8e60882ca761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:56.011361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-305eae2d-a6ca-4767-880c-8e60882ca761 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:56.012492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-305eae2d-a6ca-4767-880c-8e60882ca761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:56.064365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:56.107760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 743/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:31:56.147494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2667f-5ec4-4b46-9404-103d3e44d6d2 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:56.149017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9ef2667f-5ec4-4b46-9404-103d3e44d6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:56.184776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:56.259645 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2667f-5ec4-4b46-9404-103d3e44d6d2 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:56.334466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a3ede-1c6d-4b47-8dbf-f1dd3d9677a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:56.335017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a3ede-1c6d-4b47-8dbf-f1dd3d9677a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:56.335331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a3ede-1c6d-4b47-8dbf-f1dd3d9677a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:56.335613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a3ede-1c6d-4b47-8dbf-f1dd3d9677a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:56.337230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-387a3ede-1c6d-4b47-8dbf-f1dd3d9677a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:56.418532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:56.448991 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a3ede-1c6d-4b47-8dbf-f1dd3d9677a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:56.450727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a3ede-1c6d-4b47-8dbf-f1dd3d9677a1 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:56.469944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 747/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:31:58.656671 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e4c2e-4be4-404b-81ca-f15a561f337e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:58.657232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e4c2e-4be4-404b-81ca-f15a561f337e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:58.657232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e4c2e-4be4-404b-81ca-f15a561f337e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:58.659263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 744/1491] 10.222.0.40 () {54 vars in 926 bytes} [Mon Aug 1 07:31:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:31:58.664447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72533ae4-876c-4c3b-af1b-e02b4dbac8cc None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:58.664890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72533ae4-876c-4c3b-af1b-e02b4dbac8cc None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:58.665006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72533ae4-876c-4c3b-af1b-e02b4dbac8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:58.729945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-72533ae4-876c-4c3b-af1b-e02b4dbac8cc None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:31:58.731079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-72533ae4-876c-4c3b-af1b-e02b4dbac8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:58.826363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 748/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Aug 1 07:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:31:58.865973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e4c2e-4be4-404b-81ca-f15a561f337e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:31:58.867533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fa6e4c2e-4be4-404b-81ca-f15a561f337e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:58.972834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e4c2e-4be4-404b-81ca-f15a561f337e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:31:59.047838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd438e7-7ecf-4fb7-b8f9-87f242d32172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:31:59.048937 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd438e7-7ecf-4fb7-b8f9-87f242d32172 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:31:59.049242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd438e7-7ecf-4fb7-b8f9-87f242d32172 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:31:59.049515 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd438e7-7ecf-4fb7-b8f9-87f242d32172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:31:59.051206 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0bd438e7-7ecf-4fb7-b8f9-87f242d32172 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:31:59.060310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:31:59.160406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd438e7-7ecf-4fb7-b8f9-87f242d32172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:31:59.161845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd438e7-7ecf-4fb7-b8f9-87f242d32172 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:31:59.179484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 745/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:31:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:32:08.824541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=82 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:32:09.177172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=50 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:33:26.622350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29622a9b-5718-4a93-b034-7d8e93c87c24 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:26.622949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29622a9b-5718-4a93-b034-7d8e93c87c24 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:26.622949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29622a9b-5718-4a93-b034-7d8e93c87c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:26.699215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-29622a9b-5718-4a93-b034-7d8e93c87c24 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:33:26.700434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-29622a9b-5718-4a93-b034-7d8e93c87c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:26.795848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 749/1494] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:33:26.804591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2778cba-dd92-4e69-90c0-97df86b286d9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:26.805082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2778cba-dd92-4e69-90c0-97df86b286d9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:26.805187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2778cba-dd92-4e69-90c0-97df86b286d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:26.880786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f2778cba-dd92-4e69-90c0-97df86b286d9 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:33:26.881924 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f2778cba-dd92-4e69-90c0-97df86b286d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:26.979287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 746/1495] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:33:26.984040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29622a9b-5718-4a93-b034-7d8e93c87c24 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:33:26.985827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-29622a9b-5718-4a93-b034-7d8e93c87c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:26.997304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29622a9b-5718-4a93-b034-7d8e93c87c24 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:33:27.062248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:33:27.095589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c9647-d2c1-4d66-becd-88278a0de179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:33:27.096272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c9647-d2c1-4d66-becd-88278a0de179 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:27.096535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c9647-d2c1-4d66-becd-88278a0de179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:27.096805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c9647-d2c1-4d66-becd-88278a0de179 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:27.097484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c9647-d2c1-4d66-becd-88278a0de179 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:33:27.097785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c9647-d2c1-4d66-becd-88278a0de179 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:33:27.098799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-247c9647-d2c1-4d66-becd-88278a0de179 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:33:27.098799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 01 07:33:27.098799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: Aug 01 07:33:27.099706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c9647-d2c1-4d66-becd-88278a0de179 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:33:27.111795 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 750/1496] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:33:27.120063 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2778cba-dd92-4e69-90c0-97df86b286d9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:33:27.129270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2778cba-dd92-4e69-90c0-97df86b286d9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:33:27.190003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:33:27.223247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d7565-c512-4c65-8d03-29ed924aa3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:33:27.223863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d7565-c512-4c65-8d03-29ed924aa3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:27.224138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d7565-c512-4c65-8d03-29ed924aa3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:27.224409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d7565-c512-4c65-8d03-29ed924aa3e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:27.225056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d7565-c512-4c65-8d03-29ed924aa3e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:33:27.226771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-8c1d7565-c512-4c65-8d03-29ed924aa3e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:33:27.226771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 01 07:33:27.226771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: Aug 01 07:33:27.227987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d7565-c512-4c65-8d03-29ed924aa3e9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:33:27.283159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 747/1497] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:33:27 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:33:27.287355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c9647-d2c1-4d66-becd-88278a0de179 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:33:27.288998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-247c9647-d2c1-4d66-becd-88278a0de179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:27.321623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c9647-d2c1-4d66-becd-88278a0de179 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:33:27.414706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9240ef-ac27-43e1-a6b0-645ee7dcc9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:33:27.415337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9240ef-ac27-43e1-a6b0-645ee7dcc9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:27.415619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9240ef-ac27-43e1-a6b0-645ee7dcc9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:27.415896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9240ef-ac27-43e1-a6b0-645ee7dcc9e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:27.416561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9240ef-ac27-43e1-a6b0-645ee7dcc9e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:33:27.417573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-cf9240ef-ac27-43e1-a6b0-645ee7dcc9e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:33:27.417573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 01 07:33:27.417573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: Aug 01 07:33:27.418782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9240ef-ac27-43e1-a6b0-645ee7dcc9e9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:33:27.460708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 751/1498] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:33:27 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:33:27.464878 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d7565-c512-4c65-8d03-29ed924aa3e9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:33:27.466394 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8c1d7565-c512-4c65-8d03-29ed924aa3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:27.526570 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d7565-c512-4c65-8d03-29ed924aa3e9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:33:27.608113 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:33:27.619710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa6fd0-646b-4777-9838-a7a87d839ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:33:27.620322 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa6fd0-646b-4777-9838-a7a87d839ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:27.620601 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa6fd0-646b-4777-9838-a7a87d839ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:27.620873 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa6fd0-646b-4777-9838-a7a87d839ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:27.621558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa6fd0-646b-4777-9838-a7a87d839ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:33:27.622558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-38aa6fd0-646b-4777-9838-a7a87d839ac0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:33:27.622558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 01 07:33:27.622558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: Aug 01 07:33:27.623474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa6fd0-646b-4777-9838-a7a87d839ac0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:33:27.636453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 748/1499] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:33:27.640522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9240ef-ac27-43e1-a6b0-645ee7dcc9e9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:33:27.642040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cf9240ef-ac27-43e1-a6b0-645ee7dcc9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:27.653207 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9240ef-ac27-43e1-a6b0-645ee7dcc9e9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:33:27.678379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:33:27.749141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a78870-03d6-48f0-a512-9395592a5f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:33:27.749728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a78870-03d6-48f0-a512-9395592a5f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:27.750138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a78870-03d6-48f0-a512-9395592a5f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:27.750424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a78870-03d6-48f0-a512-9395592a5f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:27.751133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a78870-03d6-48f0-a512-9395592a5f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:33:27.752151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-46a78870-03d6-48f0-a512-9395592a5f58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:33:27.752151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 01 07:33:27.752151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: Aug 01 07:33:27.753034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a78870-03d6-48f0-a512-9395592a5f58 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:33:27.765351 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 752/1500] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:33:27.769659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa6fd0-646b-4777-9838-a7a87d839ac0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:33:27.771193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-38aa6fd0-646b-4777-9838-a7a87d839ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:27.805094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa6fd0-646b-4777-9838-a7a87d839ac0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:33:27.899824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6080f-5e85-4b5a-82d3-b29b67243fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:33:27.900492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6080f-5e85-4b5a-82d3-b29b67243fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:27.900773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6080f-5e85-4b5a-82d3-b29b67243fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:27.901049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6080f-5e85-4b5a-82d3-b29b67243fe2 admin admin] PATH_INFO: `/v3/projects/1dc2fbbca85d474db4fe7247e1587fcd/users/5c812c03eff94921a653e40e5ba408ff/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:27.923870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6080f-5e85-4b5a-82d3-b29b67243fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc2fbbca85d474db4fe7247e1587fcd, user_id=5c812c03eff94921a653e40e5ba408ff, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:33:27.924883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-5eb6080f-5e85-4b5a-82d3-b29b67243fe2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:33:27.924883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 01 07:33:27.924883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: Aug 01 07:33:27.926809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6080f-5e85-4b5a-82d3-b29b67243fe2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:33:27.952189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 749/1501] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:33:27 2022] PUT /identity/v3/projects/1dc2fbbca85d474db4fe7247e1587fcd/users/5c812c03eff94921a653e40e5ba408ff/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:33:27.965150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e97110-8efb-4c2a-9803-b4bf78d82904 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:27.965150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e97110-8efb-4c2a-9803-b4bf78d82904 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:27.965379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e97110-8efb-4c2a-9803-b4bf78d82904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:28.023874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-20e97110-8efb-4c2a-9803-b4bf78d82904 None None] MFA Rules not processed for user `5c812c03eff94921a653e40e5ba408ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:33:28.025023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-20e97110-8efb-4c2a-9803-b4bf78d82904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:28.105145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 753/1502] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:33:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:33:28.119260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0205bb-693d-496d-8978-49af364e851e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:28.119553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0205bb-693d-496d-8978-49af364e851e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:28.119839 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0205bb-693d-496d-8978-49af364e851e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:28.121201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 750/1503] 10.222.0.40 () {54 vars in 964 bytes} [Mon Aug 1 07:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:33:28.126351 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d7bc7-57d6-4422-add9-6b92fb871623 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:28.126633 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d7bc7-57d6-4422-add9-6b92fb871623 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:28.126911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d7bc7-57d6-4422-add9-6b92fb871623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:28.158982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:33:28.202321 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-af1d7bc7-57d6-4422-add9-6b92fb871623 None None] MFA Rules not processed for user `495c489f00d74b53ac02c2cbbb323c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:33:28.204064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-af1d7bc7-57d6-4422-add9-6b92fb871623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:28.279658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 754/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Mon Aug 1 07:33:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:33:28.287762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0205bb-693d-496d-8978-49af364e851e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:33:28.289346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bc0205bb-693d-496d-8978-49af364e851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:28.339892 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:33:28.397921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0205bb-693d-496d-8978-49af364e851e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:33:28.472452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96249a36-5371-4a80-abde-8f8e950de5d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:33:28.473072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96249a36-5371-4a80-abde-8f8e950de5d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:28.473352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96249a36-5371-4a80-abde-8f8e950de5d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:28.473629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96249a36-5371-4a80-abde-8f8e950de5d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:28.475299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-96249a36-5371-4a80-abde-8f8e950de5d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:28.575607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96249a36-5371-4a80-abde-8f8e950de5d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:33:28.576997 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96249a36-5371-4a80-abde-8f8e950de5d2 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:33:28.580603 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:33:28.595816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 751/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:33:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:33:28.640861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551526c5-660a-4b40-9399-e26691085755 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:28.641320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551526c5-660a-4b40-9399-e26691085755 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:28.641599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551526c5-660a-4b40-9399-e26691085755 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:28.642873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 755/1506] 10.222.0.40 () {52 vars in 830 bytes} [Mon Aug 1 07:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:33:28.646437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96249a36-5371-4a80-abde-8f8e950de5d2 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:33:28.647682 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-96249a36-5371-4a80-abde-8f8e950de5d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:28.744425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96249a36-5371-4a80-abde-8f8e950de5d2 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:33:28.835994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:33:28.836624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:28.836828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:28.837104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] PATH_INFO: `/v3/users/5c812c03eff94921a653e40e5ba408ff` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:28.849814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c812c03eff94921a653e40e5ba408ff)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:33:28.850866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:33:28.850866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 01 07:33:28.850866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: Aug 01 07:33:28.852129 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:33:28.862460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:33:28.918829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c812c03eff94921a653e40e5ba408ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:33:28.928527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c812c03eff94921a653e40e5ba408ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:33:28.951094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c812c03eff94921a653e40e5ba408ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:33:28.951626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c812c03eff94921a653e40e5ba408ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:33:28.952154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c812c03eff94921a653e40e5ba408ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:33:28.972061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 752/1507] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:33:28 2022] DELETE /identity/v3/users/5c812c03eff94921a653e40e5ba408ff => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:33:28.976438 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468e55f-2787-4a35-b934-9cf666fc3af5 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:28.976727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468e55f-2787-4a35-b934-9cf666fc3af5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:28.977025 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468e55f-2787-4a35-b934-9cf666fc3af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:29.034941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f468e55f-2787-4a35-b934-9cf666fc3af5 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:33:29.036206 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f468e55f-2787-4a35-b934-9cf666fc3af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:29.112192 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:33:29.126090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 756/1508] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:33:29.136794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:33:29.138009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:29.238259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a11811-5610-47c5-960f-e5ab95dfc66a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:33:29.293992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:33:29.315552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdd9d5-5126-4035-8779-2cfe82158741 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:33:29.316168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd9d5-5126-4035-8779-2cfe82158741 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:29.316449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd9d5-5126-4035-8779-2cfe82158741 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:29.316724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd9d5-5126-4035-8779-2cfe82158741 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:29.318328 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d4cdd9d5-5126-4035-8779-2cfe82158741 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:29.424442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdd9d5-5126-4035-8779-2cfe82158741 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:33:29.425869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdd9d5-5126-4035-8779-2cfe82158741 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:33:29.444359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 753/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:33:29.823082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468e55f-2787-4a35-b934-9cf666fc3af5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:33:29.824362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f468e55f-2787-4a35-b934-9cf666fc3af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:33:29.836660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468e55f-2787-4a35-b934-9cf666fc3af5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:33:29.935059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:33:29.935730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:33:29.936015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:33:29.936290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] PATH_INFO: `/v3/projects/1dc2fbbca85d474db4fe7247e1587fcd` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:33:29.947681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc2fbbca85d474db4fe7247e1587fcd)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:33:29.948647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:33:29.948647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 01 07:33:29.948647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: Aug 01 07:33:29.949868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:33:30.009841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc2fbbca85d474db4fe7247e1587fcd', 'request_id': 'req-5901558a-d75f-4044-b7bb-c0702bed1f7b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:33:30.016171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc2fbbca85d474db4fe7247e1587fcd', 'request_id': 'req-5901558a-d75f-4044-b7bb-c0702bed1f7b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:33:30.030281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc2fbbca85d474db4fe7247e1587fcd', 'request_id': 'req-5901558a-d75f-4044-b7bb-c0702bed1f7b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:33:30.059227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] The token cache is being invalidate because project 1dc2fbbca85d474db4fe7247e1587fcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:33:30.059526 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:33:30.059959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5901558a-d75f-4044-b7bb-c0702bed1f7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:33:30.061399 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 757/1510] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:33:29 2022] DELETE /identity/v3/projects/1dc2fbbca85d474db4fe7247e1587fcd => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:33:39.442005 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=62 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:33:40.057159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:35:06.771656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886e27c-d7e2-4f74-b3fd-eef522af6cd1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:06.772270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886e27c-d7e2-4f74-b3fd-eef522af6cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:06.772270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886e27c-d7e2-4f74-b3fd-eef522af6cd1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:06.773852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:35:06.779408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cf705-32d7-46b9-a41b-2c691fd5db1d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:06.779822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cf705-32d7-46b9-a41b-2c691fd5db1d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:06.779976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cf705-32d7-46b9-a41b-2c691fd5db1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:06.861650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-091cf705-32d7-46b9-a41b-2c691fd5db1d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:35:06.862858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-091cf705-32d7-46b9-a41b-2c691fd5db1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:06.966713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:35:07.122509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886e27c-d7e2-4f74-b3fd-eef522af6cd1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:35:07.124699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f886e27c-d7e2-4f74-b3fd-eef522af6cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:07.246260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886e27c-d7e2-4f74-b3fd-eef522af6cd1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:35:07.251790 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6973b-8e4b-4765-a5ff-56fde7305af7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:35:07.252405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6973b-8e4b-4765-a5ff-56fde7305af7 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:07.252693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6973b-8e4b-4765-a5ff-56fde7305af7 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:07.252970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6973b-8e4b-4765-a5ff-56fde7305af7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:07.261384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6973b-8e4b-4765-a5ff-56fde7305af7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:35:07.262993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6973b-8e4b-4765-a5ff-56fde7305af7 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:35:07.272201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 755/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:35:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:35:08.114462 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f684d-e057-4f19-baf4-bcb16ad50888 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:08.114987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f684d-e057-4f19-baf4-bcb16ad50888 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:08.114987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f684d-e057-4f19-baf4-bcb16ad50888 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:08.116338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:35:08.121959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c73e6d-621e-4df2-84cd-550b17afd57c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:08.122379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c73e6d-621e-4df2-84cd-550b17afd57c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:08.122536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c73e6d-621e-4df2-84cd-550b17afd57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:08.144160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-01c73e6d-621e-4df2-84cd-550b17afd57c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:35:08.145402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-01c73e6d-621e-4df2-84cd-550b17afd57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:08.153666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:35:08.309156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f684d-e057-4f19-baf4-bcb16ad50888 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:35:08.324092 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f684d-e057-4f19-baf4-bcb16ad50888 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:35:08.329669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b71b0e-7b41-43db-a622-ace46ff3d6c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:35:08.330291 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b71b0e-7b41-43db-a622-ace46ff3d6c6 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:08.330598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b71b0e-7b41-43db-a622-ace46ff3d6c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:08.330849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b71b0e-7b41-43db-a622-ace46ff3d6c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:08.339980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b71b0e-7b41-43db-a622-ace46ff3d6c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:35:08.341604 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b71b0e-7b41-43db-a622-ace46ff3d6c6 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:35:08.351579 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 760/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:35:08.805772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be3a7f-715d-4b21-a226-6089c988fd1f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:08.806563 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be3a7f-715d-4b21-a226-6089c988fd1f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:08.807140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be3a7f-715d-4b21-a226-6089c988fd1f None None] PATH_INFO: `/v3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:08.808461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 757/1517] 10.222.0.17 () {54 vars in 925 bytes} [Mon Aug 1 07:35:08 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:35:08.821454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4213a-1aad-4cdf-a7e1-9ea5c6125319 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:08.822399 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4213a-1aad-4cdf-a7e1-9ea5c6125319 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:08.822689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4213a-1aad-4cdf-a7e1-9ea5c6125319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:08.880052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2ea4213a-1aad-4cdf-a7e1-9ea5c6125319 None None] MFA Rules not processed for user `928c5d1284b74f1291c031323bb03a20`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:35:08.881507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2ea4213a-1aad-4cdf-a7e1-9ea5c6125319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:08.898573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:35:08.953674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 761/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Aug 1 07:35:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:35:12.818337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1885468-3124-4287-acc7-d3b5cefb580a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:12.818828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1885468-3124-4287-acc7-d3b5cefb580a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:12.818828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1885468-3124-4287-acc7-d3b5cefb580a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:12.820238 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 758/1519] 10.222.0.40 () {54 vars in 961 bytes} [Mon Aug 1 07:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:35:12.825584 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e614b3-f81c-4760-b40d-77d7c0cc55c7 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:12.825973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e614b3-f81c-4760-b40d-77d7c0cc55c7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:12.826152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e614b3-f81c-4760-b40d-77d7c0cc55c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:12.847928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-44e614b3-f81c-4760-b40d-77d7c0cc55c7 None None] MFA Rules not processed for user `928c5d1284b74f1291c031323bb03a20`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:35:12.849146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-44e614b3-f81c-4760-b40d-77d7c0cc55c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:12.858430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 762/1520] 10.222.0.40 () {58 vars in 1058 bytes} [Mon Aug 1 07:35:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:35:12.865970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1885468-3124-4287-acc7-d3b5cefb580a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:35:12.882465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1885468-3124-4287-acc7-d3b5cefb580a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:35:12.891278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f441b23-1e07-499e-9df8-20d88a73844d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928c5d1284b74f1291c031323bb03a20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:35:12.892354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f441b23-1e07-499e-9df8-20d88a73844d service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:12.892888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f441b23-1e07-499e-9df8-20d88a73844d service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:12.893415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f441b23-1e07-499e-9df8-20d88a73844d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:12.910390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f441b23-1e07-499e-9df8-20d88a73844d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:35:12.913604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f441b23-1e07-499e-9df8-20d88a73844d service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:35:12.927266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 759/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:35:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:35:19.588788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a1d8a-a8cb-4dd3-b096-ef9d08d98d9d None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:19.589412 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a1d8a-a8cb-4dd3-b096-ef9d08d98d9d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:19.589412 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a1d8a-a8cb-4dd3-b096-ef9d08d98d9d None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:19.590709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:35:19.595564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb926b62-d129-449c-b40e-e35e99423d31 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:19.596029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb926b62-d129-449c-b40e-e35e99423d31 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:19.596141 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb926b62-d129-449c-b40e-e35e99423d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:19.615383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cb926b62-d129-449c-b40e-e35e99423d31 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:35:19.616592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb926b62-d129-449c-b40e-e35e99423d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:19.625381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:35:19.780713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a1d8a-a8cb-4dd3-b096-ef9d08d98d9d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:35:19.795714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a1d8a-a8cb-4dd3-b096-ef9d08d98d9d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:35:19.801134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536e8f0-cc21-4942-bf3a-84394faf5468 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:35:19.801766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536e8f0-cc21-4942-bf3a-84394faf5468 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:19.802050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536e8f0-cc21-4942-bf3a-84394faf5468 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:19.802327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536e8f0-cc21-4942-bf3a-84394faf5468 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:19.811193 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536e8f0-cc21-4942-bf3a-84394faf5468 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:35:19.812772 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536e8f0-cc21-4942-bf3a-84394faf5468 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:35:19.822451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 764/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:35:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:35:20.638103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9968c-9a4f-4a79-b091-cd57f5b96870 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:20.638703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9968c-9a4f-4a79-b091-cd57f5b96870 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:20.638703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9968c-9a4f-4a79-b091-cd57f5b96870 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:20.640052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 07:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:35:20.644767 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfec156-5947-4417-9d2a-fc1dc5516d3c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:20.645220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfec156-5947-4417-9d2a-fc1dc5516d3c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:20.645331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfec156-5947-4417-9d2a-fc1dc5516d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:20.664816 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-fdfec156-5947-4417-9d2a-fc1dc5516d3c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:35:20.666151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-fdfec156-5947-4417-9d2a-fc1dc5516d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:20.674138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 07:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:35:20.825169 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9968c-9a4f-4a79-b091-cd57f5b96870 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:35:20.839813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9968c-9a4f-4a79-b091-cd57f5b96870 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:35:20.845255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bea74c-c4f8-4b57-96d8-7b2689fcf9c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:35:20.845975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bea74c-c4f8-4b57-96d8-7b2689fcf9c4 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:20.846264 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bea74c-c4f8-4b57-96d8-7b2689fcf9c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:20.846544 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bea74c-c4f8-4b57-96d8-7b2689fcf9c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:20.854831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bea74c-c4f8-4b57-96d8-7b2689fcf9c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:35:20.856495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bea74c-c4f8-4b57-96d8-7b2689fcf9c4 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:35:20.866117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 762/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:35:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:35:24.780403 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bfc03-ca56-412a-b826-e75fa888a017 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:24.780955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bfc03-ca56-412a-b826-e75fa888a017 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:24.780955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bfc03-ca56-412a-b826-e75fa888a017 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:24.782272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:35:24.787639 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94186a-46e5-4722-8149-e199cbc33680 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:24.788093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94186a-46e5-4722-8149-e199cbc33680 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:24.788227 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94186a-46e5-4722-8149-e199cbc33680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:24.808349 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1e94186a-46e5-4722-8149-e199cbc33680 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:35:24.809561 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1e94186a-46e5-4722-8149-e199cbc33680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:24.817423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:35:24.892454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bfc03-ca56-412a-b826-e75fa888a017 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:35:24.894157 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c73bfc03-ca56-412a-b826-e75fa888a017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:24.974353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bfc03-ca56-412a-b826-e75fa888a017 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:35:24.977904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fde7b4-2f96-4566-8389-eb9a6867e5b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:35:24.978567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fde7b4-2f96-4566-8389-eb9a6867e5b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:24.978845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fde7b4-2f96-4566-8389-eb9a6867e5b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:24.979148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fde7b4-2f96-4566-8389-eb9a6867e5b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:24.988032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fde7b4-2f96-4566-8389-eb9a6867e5b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:35:24.989454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fde7b4-2f96-4566-8389-eb9a6867e5b0 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:35:24.998829 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 767/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:35:26.992537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09b436-42dc-4e67-bc4c-be1d01004576 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:26.993048 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09b436-42dc-4e67-bc4c-be1d01004576 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:26.993177 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09b436-42dc-4e67-bc4c-be1d01004576 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:26.994590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:35:26.999854 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba93727-cc7b-447c-bbfe-9395733222c6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:27.000327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba93727-cc7b-447c-bbfe-9395733222c6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:27.000433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba93727-cc7b-447c-bbfe-9395733222c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:27.020297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1ba93727-cc7b-447c-bbfe-9395733222c6 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:35:27.021501 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1ba93727-cc7b-447c-bbfe-9395733222c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:27.029914 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:35:27.065637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09b436-42dc-4e67-bc4c-be1d01004576 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:35:27.076495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09b436-42dc-4e67-bc4c-be1d01004576 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:35:27.079967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e80626-6791-4997-a4c1-85782de06014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:35:27.080565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e80626-6791-4997-a4c1-85782de06014 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:27.080862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e80626-6791-4997-a4c1-85782de06014 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:27.081117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e80626-6791-4997-a4c1-85782de06014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:27.090298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e80626-6791-4997-a4c1-85782de06014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:35:27.091751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e80626-6791-4997-a4c1-85782de06014 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:35:27.100928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 765/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:35:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:35:30.676224 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d61dd4-d56e-42f1-a1c2-e45a649852a7 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:30.676794 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d61dd4-d56e-42f1-a1c2-e45a649852a7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:30.676794 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d61dd4-d56e-42f1-a1c2-e45a649852a7 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:30.678178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:35:30.684226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f974d0-4acc-4be8-9eb3-8d94cdd755e8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:30.684676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f974d0-4acc-4be8-9eb3-8d94cdd755e8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:30.684782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f974d0-4acc-4be8-9eb3-8d94cdd755e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:30.693329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=81 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:35:30.706333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-50f974d0-4acc-4be8-9eb3-8d94cdd755e8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:35:30.707568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-50f974d0-4acc-4be8-9eb3-8d94cdd755e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:30.715653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:35:31.021888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d61dd4-d56e-42f1-a1c2-e45a649852a7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:35:31.023642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-20d61dd4-d56e-42f1-a1c2-e45a649852a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:31.089100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:35:31.102489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d61dd4-d56e-42f1-a1c2-e45a649852a7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:35:31.106909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebd83a-6068-4233-a3dd-6b9b1bf9b0ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:35:31.107555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebd83a-6068-4233-a3dd-6b9b1bf9b0ba service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:31.107836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebd83a-6068-4233-a3dd-6b9b1bf9b0ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:31.108138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebd83a-6068-4233-a3dd-6b9b1bf9b0ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:31.117116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebd83a-6068-4233-a3dd-6b9b1bf9b0ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:35:31.118531 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebd83a-6068-4233-a3dd-6b9b1bf9b0ba service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:35:31.128413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 770/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:35:33.177286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7d5f6-982e-4026-ba05-4002f936ecdc None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:33.177841 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7d5f6-982e-4026-ba05-4002f936ecdc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:33.177841 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7d5f6-982e-4026-ba05-4002f936ecdc None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:33.179297 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 07:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:35:33.185313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa35a2-b73a-4b9e-966c-a680cc97cff7 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:33.185745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa35a2-b73a-4b9e-966c-a680cc97cff7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:33.185860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa35a2-b73a-4b9e-966c-a680cc97cff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:33.205069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a5aa35a2-b73a-4b9e-966c-a680cc97cff7 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:35:33.206267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a5aa35a2-b73a-4b9e-966c-a680cc97cff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:33.214475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 07:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:35:33.519439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7d5f6-982e-4026-ba05-4002f936ecdc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:35:33.521129 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1cf7d5f6-982e-4026-ba05-4002f936ecdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:35:33.534903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7d5f6-982e-4026-ba05-4002f936ecdc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:35:33.538407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a66e7-c747-4ba7-9c12-d8d7ff6ad42d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:35:33.539008 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a66e7-c747-4ba7-9c12-d8d7ff6ad42d service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:35:33.539321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a66e7-c747-4ba7-9c12-d8d7ff6ad42d service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:35:33.539600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a66e7-c747-4ba7-9c12-d8d7ff6ad42d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:35:33.549159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a66e7-c747-4ba7-9c12-d8d7ff6ad42d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:35:33.550572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a66e7-c747-4ba7-9c12-d8d7ff6ad42d service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:35:33.560093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 768/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:35:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:35:43.200541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=11 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:35:43.556633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=8 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:04.273204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf04390-7347-4464-b609-9500a590f957 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.274327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf04390-7347-4464-b609-9500a590f957 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.274844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf04390-7347-4464-b609-9500a590f957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.301419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6cf04390-7347-4464-b609-9500a590f957 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.303007 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6cf04390-7347-4464-b609-9500a590f957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.314334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 772/1540] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:04.325142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e184a2-2c39-46a7-bd19-6010eba2c592 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.326303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e184a2-2c39-46a7-bd19-6010eba2c592 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.326303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e184a2-2c39-46a7-bd19-6010eba2c592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.356126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c9e184a2-2c39-46a7-bd19-6010eba2c592 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.357338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c9e184a2-2c39-46a7-bd19-6010eba2c592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.376539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 769/1541] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.385419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf04390-7347-4464-b609-9500a590f957 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:04.399484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf04390-7347-4464-b609-9500a590f957 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:04.407183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a9865-4fe8-4305-9cf2-9008d6019f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:04.407813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a9865-4fe8-4305-9cf2-9008d6019f76 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.408099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a9865-4fe8-4305-9cf2-9008d6019f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.408383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a9865-4fe8-4305-9cf2-9008d6019f76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.409158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a9865-4fe8-4305-9cf2-9008d6019f76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:04.409475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a9865-4fe8-4305-9cf2-9008d6019f76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:04.411093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a9865-4fe8-4305-9cf2-9008d6019f76 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:04.430342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 773/1542] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:04.438551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e184a2-2c39-46a7-bd19-6010eba2c592 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:04.449373 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e184a2-2c39-46a7-bd19-6010eba2c592 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:04.457685 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9af57f-bfdf-448d-957f-82ad3f1ced41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:04.458565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9af57f-bfdf-448d-957f-82ad3f1ced41 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.459110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9af57f-bfdf-448d-957f-82ad3f1ced41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.459614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9af57f-bfdf-448d-957f-82ad3f1ced41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.460568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9af57f-bfdf-448d-957f-82ad3f1ced41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:04.462688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9af57f-bfdf-448d-957f-82ad3f1ced41 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:04.516244 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 770/1543] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:04.520967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a9865-4fe8-4305-9cf2-9008d6019f76 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:04.536827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a9865-4fe8-4305-9cf2-9008d6019f76 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:04.551576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6604b-1f5b-4fe1-8a38-edafeb00886f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.552218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6604b-1f5b-4fe1-8a38-edafeb00886f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.552715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6604b-1f5b-4fe1-8a38-edafeb00886f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.581036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e4e6604b-1f5b-4fe1-8a38-edafeb00886f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.582614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e4e6604b-1f5b-4fe1-8a38-edafeb00886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.636933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72defa3-4404-435e-8e53-4838c9627efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:04.637748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72defa3-4404-435e-8e53-4838c9627efd admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.638038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72defa3-4404-435e-8e53-4838c9627efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.638317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72defa3-4404-435e-8e53-4838c9627efd admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.643090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72defa3-4404-435e-8e53-4838c9627efd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:04.643090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72defa3-4404-435e-8e53-4838c9627efd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:04.676863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 771/1544] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:04.679287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ca2f9-29a0-4093-8522-26bd1af88a60 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.679944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ca2f9-29a0-4093-8522-26bd1af88a60 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.680223 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 774/1545] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:04.680708 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ca2f9-29a0-4093-8522-26bd1af88a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.688898 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1ad1f-6787-4fdc-96c6-0a8eb3633f83 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.689203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1ad1f-6787-4fdc-96c6-0a8eb3633f83 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.689621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1ad1f-6787-4fdc-96c6-0a8eb3633f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.708784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3a1ca2f9-29a0-4093-8522-26bd1af88a60 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.710581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3a1ca2f9-29a0-4093-8522-26bd1af88a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.726909 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 772/1546] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.728688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885b564-bcd1-424b-b0be-4e5a10a78408 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.729213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885b564-bcd1-424b-b0be-4e5a10a78408 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.729708 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885b564-bcd1-424b-b0be-4e5a10a78408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.732591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-57c1ad1f-6787-4fdc-96c6-0a8eb3633f83 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.736384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-57c1ad1f-6787-4fdc-96c6-0a8eb3633f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.748260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 775/1547] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.749255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1ad1f-6787-4fdc-96c6-0a8eb3633f83 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:04.753814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4885b564-bcd1-424b-b0be-4e5a10a78408 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.755010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4885b564-bcd1-424b-b0be-4e5a10a78408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.759536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1ad1f-6787-4fdc-96c6-0a8eb3633f83 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:04.763796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 773/1548] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.764165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043bb2d-7a78-4b2d-887c-89c87929676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:04.764767 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043bb2d-7a78-4b2d-887c-89c87929676a admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.764908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3d2f0-dd6f-4b0b-968f-8062e57c94ff None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.765082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043bb2d-7a78-4b2d-887c-89c87929676a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.765190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3d2f0-dd6f-4b0b-968f-8062e57c94ff None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.765368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3d2f0-dd6f-4b0b-968f-8062e57c94ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.765477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043bb2d-7a78-4b2d-887c-89c87929676a admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.766016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043bb2d-7a78-4b2d-887c-89c87929676a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:04.767656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043bb2d-7a78-4b2d-887c-89c87929676a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:04.778710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 776/1549] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:04.779821 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34408d34-1256-4805-ba54-b9cf84e7102c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.780116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34408d34-1256-4805-ba54-b9cf84e7102c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.780400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34408d34-1256-4805-ba54-b9cf84e7102c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.785100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-78a3d2f0-dd6f-4b0b-968f-8062e57c94ff None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.786309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-78a3d2f0-dd6f-4b0b-968f-8062e57c94ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.795417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 774/1550] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.796670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8421a1-84d2-42ff-8305-9fc0cc46c0ba None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.796965 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8421a1-84d2-42ff-8305-9fc0cc46c0ba None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.797279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8421a1-84d2-42ff-8305-9fc0cc46c0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.801410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-34408d34-1256-4805-ba54-b9cf84e7102c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.802621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-34408d34-1256-4805-ba54-b9cf84e7102c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.811902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 777/1551] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.813182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbf15a-3599-491d-ae9f-9bf0bef5bed2 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.813470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbf15a-3599-491d-ae9f-9bf0bef5bed2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.813746 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbf15a-3599-491d-ae9f-9bf0bef5bed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.822142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5a8421a1-84d2-42ff-8305-9fc0cc46c0ba None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.823499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5a8421a1-84d2-42ff-8305-9fc0cc46c0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.831679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 775/1552] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.832955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2fac7-02c2-4d25-8c5e-8a61c8641d9d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.833194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2fac7-02c2-4d25-8c5e-8a61c8641d9d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.833476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2fac7-02c2-4d25-8c5e-8a61c8641d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.833834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ddbbf15a-3599-491d-ae9f-9bf0bef5bed2 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.835212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ddbbf15a-3599-491d-ae9f-9bf0bef5bed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.845406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 778/1553] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.846452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbf15a-3599-491d-ae9f-9bf0bef5bed2 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:04.856226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0ca2fac7-02c2-4d25-8c5e-8a61c8641d9d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.857156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbf15a-3599-491d-ae9f-9bf0bef5bed2 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:04.857447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0ca2fac7-02c2-4d25-8c5e-8a61c8641d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.861767 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb938e0-80d9-4b18-a8f9-c6fa85d6eb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:04.862402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb938e0-80d9-4b18-a8f9-c6fa85d6eb6b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.862684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb938e0-80d9-4b18-a8f9-c6fa85d6eb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.862979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb938e0-80d9-4b18-a8f9-c6fa85d6eb6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.863708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb938e0-80d9-4b18-a8f9-c6fa85d6eb6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:04.864017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb938e0-80d9-4b18-a8f9-c6fa85d6eb6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:04.865504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 776/1554] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.865697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb938e0-80d9-4b18-a8f9-c6fa85d6eb6b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:04.866956 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa9127-9a0b-4f3f-9cd6-60cc70eeb5d7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.867380 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa9127-9a0b-4f3f-9cd6-60cc70eeb5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.867559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa9127-9a0b-4f3f-9cd6-60cc70eeb5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.885955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 779/1555] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:04.885955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb938e0-80d9-4b18-a8f9-c6fa85d6eb6b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:04.890152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb938e0-80d9-4b18-a8f9-c6fa85d6eb6b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:04.892228 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-61aa9127-9a0b-4f3f-9cd6-60cc70eeb5d7 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.893435 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-61aa9127-9a0b-4f3f-9cd6-60cc70eeb5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.895050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c72db1-73e8-47b3-a21e-ccf1aaa0ee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:04.895729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c72db1-73e8-47b3-a21e-ccf1aaa0ee68 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.896015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c72db1-73e8-47b3-a21e-ccf1aaa0ee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.896295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c72db1-73e8-47b3-a21e-ccf1aaa0ee68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.896980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c72db1-73e8-47b3-a21e-ccf1aaa0ee68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:04.899231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c72db1-73e8-47b3-a21e-ccf1aaa0ee68 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:04.902186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 777/1556] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.903407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d553e-a215-4b89-b9f5-b35cbd1b44a4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.903694 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d553e-a215-4b89-b9f5-b35cbd1b44a4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.903972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d553e-a215-4b89-b9f5-b35cbd1b44a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.914324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 780/1557] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:04.914324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981f46b-f356-42bf-9875-d17e751aaf56 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.914324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981f46b-f356-42bf-9875-d17e751aaf56 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.914324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981f46b-f356-42bf-9875-d17e751aaf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.925519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ba4d553e-a215-4b89-b9f5-b35cbd1b44a4 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.926788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ba4d553e-a215-4b89-b9f5-b35cbd1b44a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.932002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d981f46b-f356-42bf-9875-d17e751aaf56 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.933171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d981f46b-f356-42bf-9875-d17e751aaf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.935325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 778/1558] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.936529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6b56-a580-4ca5-9419-9910e49e5a77 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.936807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6b56-a580-4ca5-9419-9910e49e5a77 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.937091 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6b56-a580-4ca5-9419-9910e49e5a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.948869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 781/1559] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.948869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c59cc-fa3e-436e-a163-94383a7cf70e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.948869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c59cc-fa3e-436e-a163-94383a7cf70e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.948869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c59cc-fa3e-436e-a163-94383a7cf70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.961355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-265a6b56-a580-4ca5-9419-9910e49e5a77 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.962592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-265a6b56-a580-4ca5-9419-9910e49e5a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.967360 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e58c59cc-fa3e-436e-a163-94383a7cf70e None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:04.968547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e58c59cc-fa3e-436e-a163-94383a7cf70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:04.970981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 779/1560] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.971805 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6b56-a580-4ca5-9419-9910e49e5a77 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:04.977075 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 782/1561] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:04.977764 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c59cc-fa3e-436e-a163-94383a7cf70e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:04.981625 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6b56-a580-4ca5-9419-9910e49e5a77 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:04.986248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e19d57-4a1d-42ab-ab53-cd1c7233c542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:04.986863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e19d57-4a1d-42ab-ab53-cd1c7233c542 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.987184 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e19d57-4a1d-42ab-ab53-cd1c7233c542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.987468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e19d57-4a1d-42ab-ab53-cd1c7233c542 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.988158 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e19d57-4a1d-42ab-ab53-cd1c7233c542 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:04.988470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e19d57-4a1d-42ab-ab53-cd1c7233c542 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:04.988579 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c59cc-fa3e-436e-a163-94383a7cf70e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:04.989495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-c8e19d57-4a1d-42ab-ab53-cd1c7233c542 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:37:04.989495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 01 07:37:04.989495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: Aug 01 07:37:04.990472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e19d57-4a1d-42ab-ab53-cd1c7233c542 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:04.993744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c09fd-c546-4378-9bc8-f66459b4055c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:04.994344 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c09fd-c546-4378-9bc8-f66459b4055c admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:04.994630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c09fd-c546-4378-9bc8-f66459b4055c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:04.994917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c09fd-c546-4378-9bc8-f66459b4055c admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:04.995617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c09fd-c546-4378-9bc8-f66459b4055c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:04.996631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-550c09fd-c546-4378-9bc8-f66459b4055c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:37:04.996631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 01 07:37:04.996631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: Aug 01 07:37:04.997820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c09fd-c546-4378-9bc8-f66459b4055c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.003279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 780/1562] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:05.004074 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e19d57-4a1d-42ab-ab53-cd1c7233c542 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.014471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e19d57-4a1d-42ab-ab53-cd1c7233c542 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.019590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4a71f-ec5e-4ec8-91c9-fa644e5ddcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.020207 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4a71f-ec5e-4ec8-91c9-fa644e5ddcbd admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.020493 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4a71f-ec5e-4ec8-91c9-fa644e5ddcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.020773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4a71f-ec5e-4ec8-91c9-fa644e5ddcbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.021646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4a71f-ec5e-4ec8-91c9-fa644e5ddcbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:05.022097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4a71f-ec5e-4ec8-91c9-fa644e5ddcbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.023784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4a71f-ec5e-4ec8-91c9-fa644e5ddcbd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.028444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 783/1563] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:04 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.029323 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c09fd-c546-4378-9bc8-f66459b4055c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.043730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 781/1564] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:05.043730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4a71f-ec5e-4ec8-91c9-fa644e5ddcbd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.052826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4a71f-ec5e-4ec8-91c9-fa644e5ddcbd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.081015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:05.097085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:05.104504 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c09fd-c546-4378-9bc8-f66459b4055c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.109151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e62e6e-5148-4445-a9e2-212b7a896665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.109892 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e62e6e-5148-4445-a9e2-212b7a896665 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.110103 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e62e6e-5148-4445-a9e2-212b7a896665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.110379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e62e6e-5148-4445-a9e2-212b7a896665 admin admin] PATH_INFO: `/v3/projects/311edf4e3e534746bed2ff3a2888434d/users/5958b9f3a2c4424db419e7b1c667ca92/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.113501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320856b5-6498-4490-8184-5115f1561230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.114121 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320856b5-6498-4490-8184-5115f1561230 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.114409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320856b5-6498-4490-8184-5115f1561230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.114686 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320856b5-6498-4490-8184-5115f1561230 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.115414 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320856b5-6498-4490-8184-5115f1561230 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:05.115725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320856b5-6498-4490-8184-5115f1561230 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.118025 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320856b5-6498-4490-8184-5115f1561230 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.125580 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e62e6e-5148-4445-a9e2-212b7a896665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=311edf4e3e534746bed2ff3a2888434d, user_id=5958b9f3a2c4424db419e7b1c667ca92, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.126613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-98e62e6e-5148-4445-a9e2-212b7a896665 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:37:05.126613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 01 07:37:05.126613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: Aug 01 07:37:05.128576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e62e6e-5148-4445-a9e2-212b7a896665 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.129633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 782/1565] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:05.130779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320856b5-6498-4490-8184-5115f1561230 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.143167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 784/1566] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:05 2022] PUT /identity/v3/projects/311edf4e3e534746bed2ff3a2888434d/users/5958b9f3a2c4424db419e7b1c667ca92/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:05.143975 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e62e6e-5148-4445-a9e2-212b7a896665 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.151663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320856b5-6498-4490-8184-5115f1561230 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.205761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e62e6e-5148-4445-a9e2-212b7a896665 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.209149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21525fa1-9b57-4e43-ab7f-5cfe5cccd2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.210287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21525fa1-9b57-4e43-ab7f-5cfe5cccd2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.210578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21525fa1-9b57-4e43-ab7f-5cfe5cccd2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.210853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21525fa1-9b57-4e43-ab7f-5cfe5cccd2e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.211294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae713876-d5a7-4d4e-8243-6ac4b50b3dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.211585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21525fa1-9b57-4e43-ab7f-5cfe5cccd2e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:05.211894 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21525fa1-9b57-4e43-ab7f-5cfe5cccd2e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.211995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae713876-d5a7-4d4e-8243-6ac4b50b3dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.212182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae713876-d5a7-4d4e-8243-6ac4b50b3dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.212467 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae713876-d5a7-4d4e-8243-6ac4b50b3dc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.213161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae713876-d5a7-4d4e-8243-6ac4b50b3dc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:05.213470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae713876-d5a7-4d4e-8243-6ac4b50b3dc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.213591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21525fa1-9b57-4e43-ab7f-5cfe5cccd2e6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.215084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae713876-d5a7-4d4e-8243-6ac4b50b3dc8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.226619 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 783/1567] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:05.234578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 785/1568] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:05.234778 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21525fa1-9b57-4e43-ab7f-5cfe5cccd2e6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.234892 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae713876-d5a7-4d4e-8243-6ac4b50b3dc8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.238360 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae713876-d5a7-4d4e-8243-6ac4b50b3dc8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.243232 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67640246-b04c-42eb-9823-dbc3b27e4f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.243879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67640246-b04c-42eb-9823-dbc3b27e4f96 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.244171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67640246-b04c-42eb-9823-dbc3b27e4f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.244320 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21525fa1-9b57-4e43-ab7f-5cfe5cccd2e6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.244475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67640246-b04c-42eb-9823-dbc3b27e4f96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.245133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67640246-b04c-42eb-9823-dbc3b27e4f96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.247104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67640246-b04c-42eb-9823-dbc3b27e4f96 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.248955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4bed2-fdd4-45c1-9808-44c8e0ab9895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.249566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4bed2-fdd4-45c1-9808-44c8e0ab9895 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.249852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4bed2-fdd4-45c1-9808-44c8e0ab9895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.250137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4bed2-fdd4-45c1-9808-44c8e0ab9895 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.250823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4bed2-fdd4-45c1-9808-44c8e0ab9895 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:05.251166 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4bed2-fdd4-45c1-9808-44c8e0ab9895 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.252752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4bed2-fdd4-45c1-9808-44c8e0ab9895 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.265159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 784/1569] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:05.265964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4bed2-fdd4-45c1-9808-44c8e0ab9895 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.276715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4bed2-fdd4-45c1-9808-44c8e0ab9895 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.281435 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aac528-363a-4d5e-9a1e-9d5270b75d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.282041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aac528-363a-4d5e-9a1e-9d5270b75d71 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.282328 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aac528-363a-4d5e-9a1e-9d5270b75d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.282606 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aac528-363a-4d5e-9a1e-9d5270b75d71 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.283393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 786/1570] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.283890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aac528-363a-4d5e-9a1e-9d5270b75d71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.284194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67640246-b04c-42eb-9823-dbc3b27e4f96 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.284309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-b1aac528-363a-4d5e-9a1e-9d5270b75d71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:37:05.284309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 01 07:37:05.284309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: Aug 01 07:37:05.285503 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aac528-363a-4d5e-9a1e-9d5270b75d71 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.313574 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 785/1571] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/users => generated 288 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.314315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aac528-363a-4d5e-9a1e-9d5270b75d71 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.326103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aac528-363a-4d5e-9a1e-9d5270b75d71 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.350122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67640246-b04c-42eb-9823-dbc3b27e4f96 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.355297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371671a-5d55-423a-a141-114acef99fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.355913 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371671a-5d55-423a-a141-114acef99fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.356195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371671a-5d55-423a-a141-114acef99fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.356472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371671a-5d55-423a-a141-114acef99fc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.357131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371671a-5d55-423a-a141-114acef99fc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.359093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371671a-5d55-423a-a141-114acef99fc0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.385168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b7be1-9269-426a-adc6-74257db30afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.385792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b7be1-9269-426a-adc6-74257db30afe admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.386075 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b7be1-9269-426a-adc6-74257db30afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.386348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b7be1-9269-426a-adc6-74257db30afe admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.386925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 787/1572] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.387095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b7be1-9269-426a-adc6-74257db30afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.388252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c4fb0-a191-48b6-a8c1-7ba0d7934ef0 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.388537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c4fb0-a191-48b6-a8c1-7ba0d7934ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.388812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c4fb0-a191-48b6-a8c1-7ba0d7934ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.388923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b7be1-9269-426a-adc6-74257db30afe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.407542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8e8c4fb0-a191-48b6-a8c1-7ba0d7934ef0 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:05.408747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8e8c4fb0-a191-48b6-a8c1-7ba0d7934ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:05.417472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 786/1573] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.418286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b7be1-9269-426a-adc6-74257db30afe admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.424666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:05.447194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:05.474678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 788/1574] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:05.475797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c4fb0-a191-48b6-a8c1-7ba0d7934ef0 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.483696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b7be1-9269-426a-adc6-74257db30afe admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.484970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c4fb0-a191-48b6-a8c1-7ba0d7934ef0 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.488430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbf079-ef44-473e-b203-0143057b1f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.489043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbf079-ef44-473e-b203-0143057b1f77 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.489325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbf079-ef44-473e-b203-0143057b1f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.489603 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbf079-ef44-473e-b203-0143057b1f77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.489811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fde0e0-c33c-499f-97ed-5ad7144f46ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.490262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbf079-ef44-473e-b203-0143057b1f77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.490396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fde0e0-c33c-499f-97ed-5ad7144f46ab admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.490638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fde0e0-c33c-499f-97ed-5ad7144f46ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.490914 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fde0e0-c33c-499f-97ed-5ad7144f46ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.491628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fde0e0-c33c-499f-97ed-5ad7144f46ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.492241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbf079-ef44-473e-b203-0143057b1f77 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.493507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fde0e0-c33c-499f-97ed-5ad7144f46ab admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.523578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 787/1575] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.523578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbf079-ef44-473e-b203-0143057b1f77 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.523888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 789/1576] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.523888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fde0e0-c33c-499f-97ed-5ad7144f46ab admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.585581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbf079-ef44-473e-b203-0143057b1f77 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.585929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fde0e0-c33c-499f-97ed-5ad7144f46ab admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.590048 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1589c98-7997-453d-bf49-0fc98426507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.590548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396149c2-4a6e-400d-83e3-8d61e07903c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.590701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1589c98-7997-453d-bf49-0fc98426507a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.590932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1589c98-7997-453d-bf49-0fc98426507a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.591223 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396149c2-4a6e-400d-83e3-8d61e07903c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.591346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1589c98-7997-453d-bf49-0fc98426507a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.591509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396149c2-4a6e-400d-83e3-8d61e07903c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.591789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396149c2-4a6e-400d-83e3-8d61e07903c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.591903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1589c98-7997-453d-bf49-0fc98426507a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.592506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396149c2-4a6e-400d-83e3-8d61e07903c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.593782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1589c98-7997-453d-bf49-0fc98426507a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.594414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396149c2-4a6e-400d-83e3-8d61e07903c9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.615426 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 790/1577] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/users => generated 278 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.616843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396149c2-4a6e-400d-83e3-8d61e07903c9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.625370 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 788/1578] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.626602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1589c98-7997-453d-bf49-0fc98426507a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.626733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396149c2-4a6e-400d-83e3-8d61e07903c9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.683316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2573f-c880-44f0-9dab-5bb916500411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.683942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2573f-c880-44f0-9dab-5bb916500411 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.684218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2573f-c880-44f0-9dab-5bb916500411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.684493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2573f-c880-44f0-9dab-5bb916500411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.685178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2573f-c880-44f0-9dab-5bb916500411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.686789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2573f-c880-44f0-9dab-5bb916500411 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.689606 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1589c98-7997-453d-bf49-0fc98426507a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.694344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638f9cf-ce12-4d69-92e1-cf317e57c9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.694950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638f9cf-ce12-4d69-92e1-cf317e57c9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.695289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638f9cf-ce12-4d69-92e1-cf317e57c9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.695568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638f9cf-ce12-4d69-92e1-cf317e57c9c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.696376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638f9cf-ce12-4d69-92e1-cf317e57c9c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.696652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 791/1579] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:05.697472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2573f-c880-44f0-9dab-5bb916500411 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.698875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638f9cf-ce12-4d69-92e1-cf317e57c9c0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.708813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2573f-c880-44f0-9dab-5bb916500411 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.713970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409b387-1a7b-4cbb-8bc2-47e303bf25a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.714640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409b387-1a7b-4cbb-8bc2-47e303bf25a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.714953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409b387-1a7b-4cbb-8bc2-47e303bf25a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.715281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409b387-1a7b-4cbb-8bc2-47e303bf25a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.715939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409b387-1a7b-4cbb-8bc2-47e303bf25a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.717855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409b387-1a7b-4cbb-8bc2-47e303bf25a9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.724517 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 789/1580] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.724517 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638f9cf-ce12-4d69-92e1-cf317e57c9c0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.739742 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 792/1581] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/users => generated 282 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.740524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409b387-1a7b-4cbb-8bc2-47e303bf25a9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.751052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409b387-1a7b-4cbb-8bc2-47e303bf25a9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.753113 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:05.756012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971b27f-f797-4a42-81e7-a79bd1c1749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.756660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971b27f-f797-4a42-81e7-a79bd1c1749f admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.757047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971b27f-f797-4a42-81e7-a79bd1c1749f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.757415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971b27f-f797-4a42-81e7-a79bd1c1749f admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.758066 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971b27f-f797-4a42-81e7-a79bd1c1749f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.760023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971b27f-f797-4a42-81e7-a79bd1c1749f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.767807 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:05.783483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638f9cf-ce12-4d69-92e1-cf317e57c9c0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.783712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 793/1582] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/users => generated 295 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.784442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971b27f-f797-4a42-81e7-a79bd1c1749f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.787470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df068e7d-0d8c-4413-8494-526c77a49ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.788653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df068e7d-0d8c-4413-8494-526c77a49ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.788929 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df068e7d-0d8c-4413-8494-526c77a49ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.789209 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df068e7d-0d8c-4413-8494-526c77a49ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.794783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971b27f-f797-4a42-81e7-a79bd1c1749f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.798501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df068e7d-0d8c-4413-8494-526c77a49ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.799586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0315448-e850-4eec-aac7-34522014d385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.800010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df068e7d-0d8c-4413-8494-526c77a49ac6 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.800207 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315448-e850-4eec-aac7-34522014d385 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.800518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315448-e850-4eec-aac7-34522014d385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.800792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315448-e850-4eec-aac7-34522014d385 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.801444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0315448-e850-4eec-aac7-34522014d385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.803366 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0315448-e850-4eec-aac7-34522014d385 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.809609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 790/1583] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:05.810415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df068e7d-0d8c-4413-8494-526c77a49ac6 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.819968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df068e7d-0d8c-4413-8494-526c77a49ac6 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.827431 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00facf-b2d2-4c5c-b6db-dead4fcacde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.828212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00facf-b2d2-4c5c-b6db-dead4fcacde1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.828503 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00facf-b2d2-4c5c-b6db-dead4fcacde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.828938 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00facf-b2d2-4c5c-b6db-dead4fcacde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.829672 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00facf-b2d2-4c5c-b6db-dead4fcacde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.833342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 794/1584] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.833481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00facf-b2d2-4c5c-b6db-dead4fcacde1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.833581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0315448-e850-4eec-aac7-34522014d385 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.843258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 791/1585] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:05.844636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0315448-e850-4eec-aac7-34522014d385 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.846857 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00facf-b2d2-4c5c-b6db-dead4fcacde1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.849570 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8675a99-142b-4d90-a843-d7e11b37254d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.850254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8675a99-142b-4d90-a843-d7e11b37254d admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.851269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8675a99-142b-4d90-a843-d7e11b37254d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.851554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8675a99-142b-4d90-a843-d7e11b37254d admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.852430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8675a99-142b-4d90-a843-d7e11b37254d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.854437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8675a99-142b-4d90-a843-d7e11b37254d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.857899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00facf-b2d2-4c5c-b6db-dead4fcacde1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.863039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b518fa-9d15-4f05-805f-6081106c8892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.863681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b518fa-9d15-4f05-805f-6081106c8892 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.863974 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b518fa-9d15-4f05-805f-6081106c8892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.864259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b518fa-9d15-4f05-805f-6081106c8892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.864963 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b518fa-9d15-4f05-805f-6081106c8892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.866559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b518fa-9d15-4f05-805f-6081106c8892 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.880699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 792/1586] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:05.880699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b518fa-9d15-4f05-805f-6081106c8892 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.883517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 795/1587] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:05 2022] POST /identity/v3/users => generated 295 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:05.884612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8675a99-142b-4d90-a843-d7e11b37254d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.888594 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b518fa-9d15-4f05-805f-6081106c8892 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.894298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2125ae-d8fb-419e-a27d-5c7f138ccf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.894919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2125ae-d8fb-419e-a27d-5c7f138ccf17 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.895295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2125ae-d8fb-419e-a27d-5c7f138ccf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.895571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2125ae-d8fb-419e-a27d-5c7f138ccf17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.896234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2125ae-d8fb-419e-a27d-5c7f138ccf17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.898290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8675a99-142b-4d90-a843-d7e11b37254d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.898607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2125ae-d8fb-419e-a27d-5c7f138ccf17 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.903343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e88f5d-f424-432b-8183-1f68749304b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.903962 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e88f5d-f424-432b-8183-1f68749304b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.904238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e88f5d-f424-432b-8183-1f68749304b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.904515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e88f5d-f424-432b-8183-1f68749304b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.905303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e88f5d-f424-432b-8183-1f68749304b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.907400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e88f5d-f424-432b-8183-1f68749304b8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.919484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 793/1588] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:05.919484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2125ae-d8fb-419e-a27d-5c7f138ccf17 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.919796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 796/1589] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:05.923593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e88f5d-f424-432b-8183-1f68749304b8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.925408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2125ae-d8fb-419e-a27d-5c7f138ccf17 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.931021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17acaf6-0d39-447f-b00d-27898df9f4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.931550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17acaf6-0d39-447f-b00d-27898df9f4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.931837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17acaf6-0d39-447f-b00d-27898df9f4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.932120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17acaf6-0d39-447f-b00d-27898df9f4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.932812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17acaf6-0d39-447f-b00d-27898df9f4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.933770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e88f5d-f424-432b-8183-1f68749304b8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.934480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17acaf6-0d39-447f-b00d-27898df9f4f5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.938840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505ff06-e652-461b-93b5-bcf49a3db013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.939590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505ff06-e652-461b-93b5-bcf49a3db013 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.939878 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505ff06-e652-461b-93b5-bcf49a3db013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.940153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505ff06-e652-461b-93b5-bcf49a3db013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.940967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505ff06-e652-461b-93b5-bcf49a3db013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.942608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505ff06-e652-461b-93b5-bcf49a3db013 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.946327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 794/1590] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:05.947428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17acaf6-0d39-447f-b00d-27898df9f4f5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.955400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 797/1591] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:05.956586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505ff06-e652-461b-93b5-bcf49a3db013 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:05.963318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17acaf6-0d39-447f-b00d-27898df9f4f5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.968950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505ff06-e652-461b-93b5-bcf49a3db013 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:05.969712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a71e0-35e0-4e2b-b0df-16b8477c80a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.970335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a71e0-35e0-4e2b-b0df-16b8477c80a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.970623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a71e0-35e0-4e2b-b0df-16b8477c80a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.970904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a71e0-35e0-4e2b-b0df-16b8477c80a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.971654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a71e0-35e0-4e2b-b0df-16b8477c80a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:05.973265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a71e0-35e0-4e2b-b0df-16b8477c80a2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:05.975280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d636196-c838-40d6-9693-6ceb7798cbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:05.976871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d636196-c838-40d6-9693-6ceb7798cbec admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:05.977333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d636196-c838-40d6-9693-6ceb7798cbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:05.977891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d636196-c838-40d6-9693-6ceb7798cbec admin admin] PATH_INFO: `/v3/projects/2f0667d27d174403b879f4c13a318fa3/users/f38a13230b0e4baab9148a2e20bbe73b/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:05.985193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 795/1592] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:05.986063 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a71e0-35e0-4e2b-b0df-16b8477c80a2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:06.001246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a71e0-35e0-4e2b-b0df-16b8477c80a2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:06.001439 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d636196-c838-40d6-9693-6ceb7798cbec admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0667d27d174403b879f4c13a318fa3, user_id=f38a13230b0e4baab9148a2e20bbe73b, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.004989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d636196-c838-40d6-9693-6ceb7798cbec admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.013251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c372b14-c81e-404e-9a99-79deea1ea39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:06.013251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c372b14-c81e-404e-9a99-79deea1ea39f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.013251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c372b14-c81e-404e-9a99-79deea1ea39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.013251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c372b14-c81e-404e-9a99-79deea1ea39f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.013966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c372b14-c81e-404e-9a99-79deea1ea39f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.016084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c372b14-c81e-404e-9a99-79deea1ea39f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.031456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 798/1593] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:05 2022] PUT /identity/v3/projects/2f0667d27d174403b879f4c13a318fa3/users/f38a13230b0e4baab9148a2e20bbe73b/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:06.035213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 796/1594] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:06.035213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c372b14-c81e-404e-9a99-79deea1ea39f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:06.037476 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d636196-c838-40d6-9693-6ceb7798cbec admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:06.049848 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c372b14-c81e-404e-9a99-79deea1ea39f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:06.111403 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d636196-c838-40d6-9693-6ceb7798cbec admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:06.117857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65ec32-5eb2-4d75-9d59-14671e51ff20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:06.118488 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65ec32-5eb2-4d75-9d59-14671e51ff20 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.118765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65ec32-5eb2-4d75-9d59-14671e51ff20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.119137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65ec32-5eb2-4d75-9d59-14671e51ff20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.119869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65ec32-5eb2-4d75-9d59-14671e51ff20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.121759 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65ec32-5eb2-4d75-9d59-14671e51ff20 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.130096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b28bc-137e-4184-81e7-0ea058937d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:06.131131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b28bc-137e-4184-81e7-0ea058937d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.131525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b28bc-137e-4184-81e7-0ea058937d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.132211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b28bc-137e-4184-81e7-0ea058937d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.133139 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b28bc-137e-4184-81e7-0ea058937d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.133668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 799/1595] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:06.134802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65ec32-5eb2-4d75-9d59-14671e51ff20 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:06.135500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b28bc-137e-4184-81e7-0ea058937d06 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.145634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65ec32-5eb2-4d75-9d59-14671e51ff20 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:06.149920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 797/1596] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:06.149920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b28bc-137e-4184-81e7-0ea058937d06 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:06.153338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869e41b-b61a-40f9-b78a-007dc5cc00b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:06.153910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869e41b-b61a-40f9-b78a-007dc5cc00b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.154199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869e41b-b61a-40f9-b78a-007dc5cc00b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.154480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869e41b-b61a-40f9-b78a-007dc5cc00b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.155480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869e41b-b61a-40f9-b78a-007dc5cc00b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.156837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869e41b-b61a-40f9-b78a-007dc5cc00b0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.159085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b28bc-137e-4184-81e7-0ea058937d06 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:06.164790 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec1d4a-b44f-4cce-aaf6-9fd5ac3b5045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:06.165509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec1d4a-b44f-4cce-aaf6-9fd5ac3b5045 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.165903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec1d4a-b44f-4cce-aaf6-9fd5ac3b5045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.166261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec1d4a-b44f-4cce-aaf6-9fd5ac3b5045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.167580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec1d4a-b44f-4cce-aaf6-9fd5ac3b5045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.168832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec1d4a-b44f-4cce-aaf6-9fd5ac3b5045 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.171333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 800/1597] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:06.172241 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869e41b-b61a-40f9-b78a-007dc5cc00b0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:06.184826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 798/1598] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:06.188151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869e41b-b61a-40f9-b78a-007dc5cc00b0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:06.189665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec1d4a-b44f-4cce-aaf6-9fd5ac3b5045 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:06.194020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8aa740-0e8f-469d-9a95-936306a04300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:06.195744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8aa740-0e8f-469d-9a95-936306a04300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.196158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8aa740-0e8f-469d-9a95-936306a04300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.196605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8aa740-0e8f-469d-9a95-936306a04300 admin admin] PATH_INFO: `/v3/projects/78ff3c4979364b75a0746c0e904a53fc/users/4e176fb300ae4e7b9d821ec54cf05cde/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.201995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec1d4a-b44f-4cce-aaf6-9fd5ac3b5045 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:06.207624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcc89b-e636-4983-a29c-265386051e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:06.208439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcc89b-e636-4983-a29c-265386051e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.208758 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcc89b-e636-4983-a29c-265386051e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.209123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcc89b-e636-4983-a29c-265386051e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.210104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcc89b-e636-4983-a29c-265386051e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.211993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcc89b-e636-4983-a29c-265386051e5a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.216977 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8aa740-0e8f-469d-9a95-936306a04300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ff3c4979364b75a0746c0e904a53fc, user_id=4e176fb300ae4e7b9d821ec54cf05cde, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.219881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8aa740-0e8f-469d-9a95-936306a04300 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.228382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 799/1599] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:06.228382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52a9d9-31d5-43c7-baf6-d5eff73de337 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.228382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52a9d9-31d5-43c7-baf6-d5eff73de337 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.228382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52a9d9-31d5-43c7-baf6-d5eff73de337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.241267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 801/1600] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:06 2022] PUT /identity/v3/projects/78ff3c4979364b75a0746c0e904a53fc/users/4e176fb300ae4e7b9d821ec54cf05cde/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:06.242190 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8aa740-0e8f-469d-9a95-936306a04300 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:06.270114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8b52a9d9-31d5-43c7-baf6-d5eff73de337 None None] MFA Rules not processed for user `f38a13230b0e4baab9148a2e20bbe73b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:06.271563 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b52a9d9-31d5-43c7-baf6-d5eff73de337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:06.296446 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:INSERT=11,keystone:SELECT=89 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:06.313310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:06.334891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8aa740-0e8f-469d-9a95-936306a04300 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:06.339617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacc7e4-d067-4488-9a46-36a124b0fa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:06.340358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacc7e4-d067-4488-9a46-36a124b0fa9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.340645 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacc7e4-d067-4488-9a46-36a124b0fa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.341006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacc7e4-d067-4488-9a46-36a124b0fa9e admin admin] PATH_INFO: `/v3/projects/a589760b9c674009a0ec8007d1669f12/users/e4a248d79a3f4aa2b7937e566508e559/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.359877 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacc7e4-d067-4488-9a46-36a124b0fa9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a589760b9c674009a0ec8007d1669f12, user_id=e4a248d79a3f4aa2b7937e566508e559, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.360421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 800/1601] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:06.361098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52a9d9-31d5-43c7-baf6-d5eff73de337 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:06.363314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacc7e4-d067-4488-9a46-36a124b0fa9e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.373468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52a9d9-31d5-43c7-baf6-d5eff73de337 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:06.382942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 802/1602] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:06 2022] PUT /identity/v3/projects/a589760b9c674009a0ec8007d1669f12/users/e4a248d79a3f4aa2b7937e566508e559/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:06.387365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacc7e4-d067-4488-9a46-36a124b0fa9e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:06.445994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3c899-fdb8-4953-b63e-6d7f38fc0141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:06.446679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3c899-fdb8-4953-b63e-6d7f38fc0141 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.446943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3c899-fdb8-4953-b63e-6d7f38fc0141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.447253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3c899-fdb8-4953-b63e-6d7f38fc0141 admin admin] PATH_INFO: `/v3/projects/640f07cfa0a543c8a1a4c3d0d74ceebd/users/2b52798ad058487192937fe12177cf70/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.464733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3c899-fdb8-4953-b63e-6d7f38fc0141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=640f07cfa0a543c8a1a4c3d0d74ceebd, user_id=2b52798ad058487192937fe12177cf70, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.467473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3c899-fdb8-4953-b63e-6d7f38fc0141 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.476638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacc7e4-d067-4488-9a46-36a124b0fa9e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:06.482970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 801/1603] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:06 2022] PUT /identity/v3/projects/640f07cfa0a543c8a1a4c3d0d74ceebd/users/2b52798ad058487192937fe12177cf70/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:06.483796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3c899-fdb8-4953-b63e-6d7f38fc0141 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:06.550286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3c899-fdb8-4953-b63e-6d7f38fc0141 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:06.552286 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd9147-6295-44c3-b923-015c35a8c1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:06.553008 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd9147-6295-44c3-b923-015c35a8c1d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.553263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd9147-6295-44c3-b923-015c35a8c1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.554291 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd9147-6295-44c3-b923-015c35a8c1d9 admin admin] PATH_INFO: `/v3/projects/015cc91d3e2f4d3791301c05e4c8be20/users/db6ac4cf9dd84e338f944b4905dcdb50/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.556771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecdc50-3e61-4dc2-8dff-89e03460a37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:06.557480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecdc50-3e61-4dc2-8dff-89e03460a37d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.557945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecdc50-3e61-4dc2-8dff-89e03460a37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.558240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecdc50-3e61-4dc2-8dff-89e03460a37d admin admin] PATH_INFO: `/v3/projects/53bab4dbf0c34ae0be1be397427c99e8/users/905e317fcf49428ea1fa708dfabc958e/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.573563 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd9147-6295-44c3-b923-015c35a8c1d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=015cc91d3e2f4d3791301c05e4c8be20, user_id=db6ac4cf9dd84e338f944b4905dcdb50, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.575286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecdc50-3e61-4dc2-8dff-89e03460a37d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53bab4dbf0c34ae0be1be397427c99e8, user_id=905e317fcf49428ea1fa708dfabc958e, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.576618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd9147-6295-44c3-b923-015c35a8c1d9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.578133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecdc50-3e61-4dc2-8dff-89e03460a37d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.592043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 802/1604] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:06 2022] PUT /identity/v3/projects/53bab4dbf0c34ae0be1be397427c99e8/users/905e317fcf49428ea1fa708dfabc958e/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:06.592989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecdc50-3e61-4dc2-8dff-89e03460a37d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:06.595158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 803/1605] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:06 2022] PUT /identity/v3/projects/015cc91d3e2f4d3791301c05e4c8be20/users/db6ac4cf9dd84e338f944b4905dcdb50/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:06.597104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6e73-a77a-412b-932d-4d5bc5dbd069 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.598890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6e73-a77a-412b-932d-4d5bc5dbd069 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.599806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6e73-a77a-412b-932d-4d5bc5dbd069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.638147 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:06.642388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-41be6e73-a77a-412b-932d-4d5bc5dbd069 None None] MFA Rules not processed for user `4e176fb300ae4e7b9d821ec54cf05cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:06.644092 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-41be6e73-a77a-412b-932d-4d5bc5dbd069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:06.674178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecdc50-3e61-4dc2-8dff-89e03460a37d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:06.679914 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e474f68-b4a3-42e9-8032-3ee0daac1312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:06.680596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e474f68-b4a3-42e9-8032-3ee0daac1312 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.680945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e474f68-b4a3-42e9-8032-3ee0daac1312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.681237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e474f68-b4a3-42e9-8032-3ee0daac1312 admin admin] PATH_INFO: `/v3/projects/4a2b83a4dad243e79144db87357d1897/users/1e02f414ef2a4990bd1492a63d074319/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.681665 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:06.702358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e474f68-b4a3-42e9-8032-3ee0daac1312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2b83a4dad243e79144db87357d1897, user_id=1e02f414ef2a4990bd1492a63d074319, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:06.705501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e474f68-b4a3-42e9-8032-3ee0daac1312 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:06.721931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 803/1606] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:06 2022] PUT /identity/v3/projects/4a2b83a4dad243e79144db87357d1897/users/1e02f414ef2a4990bd1492a63d074319/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:06.723530 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5242219-746f-490d-aaed-01715b7aeba5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.723841 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5242219-746f-490d-aaed-01715b7aeba5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.724120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5242219-746f-490d-aaed-01715b7aeba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.736562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 804/1607] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:06 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:06.738002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9565ea3-6e3a-4f89-a61d-737e8de2ab48 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.738352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9565ea3-6e3a-4f89-a61d-737e8de2ab48 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.738634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9565ea3-6e3a-4f89-a61d-737e8de2ab48 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.740377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 805/1608] 10.222.0.40 () {54 vars in 962 bytes} [Mon Aug 1 07:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:37:06.756073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7974a10-c35f-40d7-97a8-2f0546b1da54 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.756360 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7974a10-c35f-40d7-97a8-2f0546b1da54 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.756734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7974a10-c35f-40d7-97a8-2f0546b1da54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.759671 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d5242219-746f-490d-aaed-01715b7aeba5 None None] MFA Rules not processed for user `e4a248d79a3f4aa2b7937e566508e559`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:06.761183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d5242219-746f-490d-aaed-01715b7aeba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:06.793346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a7974a10-c35f-40d7-97a8-2f0546b1da54 None None] MFA Rules not processed for user `2b52798ad058487192937fe12177cf70`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:06.794926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a7974a10-c35f-40d7-97a8-2f0546b1da54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:06.857278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 804/1609] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:06.859173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b11d52-644d-42b6-a727-5a1710303ab7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.859407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b11d52-644d-42b6-a727-5a1710303ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.859813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b11d52-644d-42b6-a727-5a1710303ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.889346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 806/1610] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:06.891092 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a16794-10b7-448a-b270-c6fd6fed6a24 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:06.891548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a16794-10b7-448a-b270-c6fd6fed6a24 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:06.892015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a16794-10b7-448a-b270-c6fd6fed6a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:06.915798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-15a16794-10b7-448a-b270-c6fd6fed6a24 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:06.917682 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-15a16794-10b7-448a-b270-c6fd6fed6a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:06.918760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-93b11d52-644d-42b6-a727-5a1710303ab7 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:06.920052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-93b11d52-644d-42b6-a727-5a1710303ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:07.002711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 805/1611] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:07.004019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fe190-15c4-4f52-b189-e37fcdabdd79 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.004376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fe190-15c4-4f52-b189-e37fcdabdd79 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.004661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fe190-15c4-4f52-b189-e37fcdabdd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.023143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 807/1612] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:07.023143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a16794-10b7-448a-b270-c6fd6fed6a24 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:07.027099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bf0fe190-15c4-4f52-b189-e37fcdabdd79 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:07.028616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bf0fe190-15c4-4f52-b189-e37fcdabdd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:07.040390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 806/1613] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:07.041828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c027891-a255-4051-84e4-5736c413a2ff None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.042751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c027891-a255-4051-84e4-5736c413a2ff None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.043023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c027891-a255-4051-84e4-5736c413a2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.050607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a16794-10b7-448a-b270-c6fd6fed6a24 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:07.056140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cff4a1-de3b-4749-87a3-2af6f032985f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:07.056822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cff4a1-de3b-4749-87a3-2af6f032985f admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.057109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cff4a1-de3b-4749-87a3-2af6f032985f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.057386 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cff4a1-de3b-4749-87a3-2af6f032985f admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.058054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cff4a1-de3b-4749-87a3-2af6f032985f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:07.060094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cff4a1-de3b-4749-87a3-2af6f032985f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:07.089161 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3c027891-a255-4051-84e4-5736c413a2ff None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:07.090307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3c027891-a255-4051-84e4-5736c413a2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:07.104112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 808/1614] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:07 2022] POST /identity/v3/projects => generated 355 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:07.105607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cff4a1-de3b-4749-87a3-2af6f032985f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:07.124034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:07.160212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:07.185711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cff4a1-de3b-4749-87a3-2af6f032985f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:07.186382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 807/1615] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Aug 1 07:37:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:07.188278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2a8e3-c1fb-446e-8f1e-a25586602ae7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.188742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2a8e3-c1fb-446e-8f1e-a25586602ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.189193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2a8e3-c1fb-446e-8f1e-a25586602ae7 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.190969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 808/1616] 10.222.0.40 () {54 vars in 962 bytes} [Mon Aug 1 07:37:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:37:07.191710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c11c76-43ea-42a3-9542-c8023df5d429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:07.192234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2a8e3-c1fb-446e-8f1e-a25586602ae7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:07.192592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c11c76-43ea-42a3-9542-c8023df5d429 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.193018 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c11c76-43ea-42a3-9542-c8023df5d429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.193395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c11c76-43ea-42a3-9542-c8023df5d429 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.194226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c11c76-43ea-42a3-9542-c8023df5d429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:07.195274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-93d2a8e3-c1fb-446e-8f1e-a25586602ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:07.196768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c11c76-43ea-42a3-9542-c8023df5d429 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:07.234764 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 809/1617] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:07 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:07.235849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c11c76-43ea-42a3-9542-c8023df5d429 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:07.238985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-24c11c76-43ea-42a3-9542-c8023df5d429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:07.297957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2a8e3-c1fb-446e-8f1e-a25586602ae7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:07.315311 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c11c76-43ea-42a3-9542-c8023df5d429 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:07.321925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de702f-752b-46fa-893a-145726aba602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:07.322779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de702f-752b-46fa-893a-145726aba602 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.323139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de702f-752b-46fa-893a-145726aba602 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.323433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de702f-752b-46fa-893a-145726aba602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.368551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d8537-bc0a-4e79-b9c8-7723f48c429f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:07.369714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d8537-bc0a-4e79-b9c8-7723f48c429f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.370141 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d8537-bc0a-4e79-b9c8-7723f48c429f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.370719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d8537-bc0a-4e79-b9c8-7723f48c429f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.421046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de702f-752b-46fa-893a-145726aba602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:07.422715 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de702f-752b-46fa-893a-145726aba602 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:07.441964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 810/1618] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:07.443403 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de702f-752b-46fa-893a-145726aba602 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:07.461842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de702f-752b-46fa-893a-145726aba602 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:07.466633 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef32ba-0dac-4690-a4cf-313f2cff1ad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:07.467092 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef32ba-0dac-4690-a4cf-313f2cff1ad9 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.467397 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef32ba-0dac-4690-a4cf-313f2cff1ad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.467688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef32ba-0dac-4690-a4cf-313f2cff1ad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.485782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef32ba-0dac-4690-a4cf-313f2cff1ad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:07.487280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef32ba-0dac-4690-a4cf-313f2cff1ad9 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:07.496300 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d8537-bc0a-4e79-b9c8-7723f48c429f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:07.498439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d8537-bc0a-4e79-b9c8-7723f48c429f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:07.499105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 811/1619] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:07.499912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef32ba-0dac-4690-a4cf-313f2cff1ad9 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:07.513849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef32ba-0dac-4690-a4cf-313f2cff1ad9 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:07.525223 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577cd2b-5397-4575-a1cf-fc0afcfb3d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:07.526010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577cd2b-5397-4575-a1cf-fc0afcfb3d18 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.526978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577cd2b-5397-4575-a1cf-fc0afcfb3d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.527356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577cd2b-5397-4575-a1cf-fc0afcfb3d18 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.530043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 809/1620] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:07.530765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577cd2b-5397-4575-a1cf-fc0afcfb3d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:07.534180 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66047b6-80c6-4ed1-b755-07b2bc685ef1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.534568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66047b6-80c6-4ed1-b755-07b2bc685ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.534802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577cd2b-5397-4575-a1cf-fc0afcfb3d18 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:07.535286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66047b6-80c6-4ed1-b755-07b2bc685ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.579624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 812/1621] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:07 2022] POST /identity/v3/users => generated 284 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:07.579624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577cd2b-5397-4575-a1cf-fc0afcfb3d18 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:07.583265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c66047b6-80c6-4ed1-b755-07b2bc685ef1 None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:07.585177 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c66047b6-80c6-4ed1-b755-07b2bc685ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:07.610891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:07.662314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:07.709190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 810/1622] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Aug 1 07:37:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:07.711337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66047b6-80c6-4ed1-b755-07b2bc685ef1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:07.719196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577cd2b-5397-4575-a1cf-fc0afcfb3d18 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:07.727285 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1135a03-a77d-406d-b29c-b6681062e9a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:07.728252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1135a03-a77d-406d-b29c-b6681062e9a0 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.728705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1135a03-a77d-406d-b29c-b6681062e9a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.729330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1135a03-a77d-406d-b29c-b6681062e9a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.737926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66047b6-80c6-4ed1-b755-07b2bc685ef1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:07.745141 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6054d57-a020-47ad-b975-18de2949edeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:07.746089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6054d57-a020-47ad-b975-18de2949edeb admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.746697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6054d57-a020-47ad-b975-18de2949edeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.747188 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6054d57-a020-47ad-b975-18de2949edeb admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.748148 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6054d57-a020-47ad-b975-18de2949edeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:07.750400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6054d57-a020-47ad-b975-18de2949edeb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:07.843628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 811/1623] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:07 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:07.843628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6054d57-a020-47ad-b975-18de2949edeb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:07.849870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1135a03-a77d-406d-b29c-b6681062e9a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:07.849870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1135a03-a77d-406d-b29c-b6681062e9a0 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:07.850911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6054d57-a020-47ad-b975-18de2949edeb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:07.854642 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ce003-6027-4b7f-99ee-dae908196dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:07.855536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ce003-6027-4b7f-99ee-dae908196dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.855982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ce003-6027-4b7f-99ee-dae908196dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.856445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ce003-6027-4b7f-99ee-dae908196dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.857657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ce003-6027-4b7f-99ee-dae908196dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:07.862395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ce003-6027-4b7f-99ee-dae908196dbf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:07.863836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 813/1624] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:07.868018 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1135a03-a77d-406d-b29c-b6681062e9a0 service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:07.887586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1135a03-a77d-406d-b29c-b6681062e9a0 service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:07.896509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaee6f4-4d87-4acf-aedc-5acc3046928e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:07.897285 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaee6f4-4d87-4acf-aedc-5acc3046928e service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.897592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaee6f4-4d87-4acf-aedc-5acc3046928e service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.899176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaee6f4-4d87-4acf-aedc-5acc3046928e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.922482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 812/1625] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:07.923755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ce003-6027-4b7f-99ee-dae908196dbf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:07.960584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ce003-6027-4b7f-99ee-dae908196dbf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:07.972535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9856be-de32-4324-83b2-4e7e59f48b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:07.976048 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9856be-de32-4324-83b2-4e7e59f48b49 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:07.979657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9856be-de32-4324-83b2-4e7e59f48b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:07.979977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9856be-de32-4324-83b2-4e7e59f48b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:07.980735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9856be-de32-4324-83b2-4e7e59f48b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:07.987760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9856be-de32-4324-83b2-4e7e59f48b49 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:08.014842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 813/1626] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:08.015931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9856be-de32-4324-83b2-4e7e59f48b49 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:08.032175 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaee6f4-4d87-4acf-aedc-5acc3046928e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:08.047347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9856be-de32-4324-83b2-4e7e59f48b49 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:08.056073 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d1019-cbfc-40ea-a61d-d52257f19f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:08.057871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d1019-cbfc-40ea-a61d-d52257f19f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:08.058261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d1019-cbfc-40ea-a61d-d52257f19f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:08.058711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d1019-cbfc-40ea-a61d-d52257f19f85 admin admin] PATH_INFO: `/v3/projects/df73b49fde7a48108a4cb6e6b36957aa/users/c7bd3cafc6af4a0980701a3327b78f56/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:08.065199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaee6f4-4d87-4acf-aedc-5acc3046928e service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:08.078503 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d1019-cbfc-40ea-a61d-d52257f19f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df73b49fde7a48108a4cb6e6b36957aa, user_id=c7bd3cafc6af4a0980701a3327b78f56, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:08.079285 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 814/1627] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:08.080288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaee6f4-4d87-4acf-aedc-5acc3046928e service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:08.082562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d1019-cbfc-40ea-a61d-d52257f19f85 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:08.095927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaee6f4-4d87-4acf-aedc-5acc3046928e service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:08.105534 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 814/1628] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:08 2022] PUT /identity/v3/projects/df73b49fde7a48108a4cb6e6b36957aa/users/c7bd3cafc6af4a0980701a3327b78f56/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:08.109635 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d1019-cbfc-40ea-a61d-d52257f19f85 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:08.188243 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91864f50-d975-4758-89b4-f2663296c28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:08.189597 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91864f50-d975-4758-89b4-f2663296c28b admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:08.191105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91864f50-d975-4758-89b4-f2663296c28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:08.191604 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91864f50-d975-4758-89b4-f2663296c28b admin admin] PATH_INFO: `/v3/projects/3498c30e42ef46b884d8c2d607067b12/users/0b6a3496e6194af28e51fabdeefabd30/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:08.212729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91864f50-d975-4758-89b4-f2663296c28b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3498c30e42ef46b884d8c2d607067b12, user_id=0b6a3496e6194af28e51fabdeefabd30, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:08.217391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91864f50-d975-4758-89b4-f2663296c28b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:08.240547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 815/1629] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:08 2022] PUT /identity/v3/projects/3498c30e42ef46b884d8c2d607067b12/users/0b6a3496e6194af28e51fabdeefabd30/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:08.245023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91864f50-d975-4758-89b4-f2663296c28b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:08.250980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:08.320767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d1019-cbfc-40ea-a61d-d52257f19f85 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:08.391995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91864f50-d975-4758-89b4-f2663296c28b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:08.418171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e3e85-d13e-4a34-aa57-ac54293fbd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:08.420063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e3e85-d13e-4a34-aa57-ac54293fbd2b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:08.421445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e3e85-d13e-4a34-aa57-ac54293fbd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:08.421753 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e3e85-d13e-4a34-aa57-ac54293fbd2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:08.422548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e3e85-d13e-4a34-aa57-ac54293fbd2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:08.426752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e3e85-d13e-4a34-aa57-ac54293fbd2b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:08.459093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 816/1630] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:08.459487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e3e85-d13e-4a34-aa57-ac54293fbd2b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:08.483187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e3e85-d13e-4a34-aa57-ac54293fbd2b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:08.484508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84c4e3-7bb9-4b62-9649-c3fae7baed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:08.485524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84c4e3-7bb9-4b62-9649-c3fae7baed27 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:08.485524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84c4e3-7bb9-4b62-9649-c3fae7baed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:08.485752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84c4e3-7bb9-4b62-9649-c3fae7baed27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:08.486367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84c4e3-7bb9-4b62-9649-c3fae7baed27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:08.488000 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84c4e3-7bb9-4b62-9649-c3fae7baed27 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:08.497448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:08.499735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904bf3b-6999-4f49-962f-dd68c8c862f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:08.500438 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904bf3b-6999-4f49-962f-dd68c8c862f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:08.500719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904bf3b-6999-4f49-962f-dd68c8c862f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:08.508324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904bf3b-6999-4f49-962f-dd68c8c862f6 admin admin] PATH_INFO: `/v3/projects/3498c30e42ef46b884d8c2d607067b12/users/0b6a3496e6194af28e51fabdeefabd30/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:08.516017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904bf3b-6999-4f49-962f-dd68c8c862f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3498c30e42ef46b884d8c2d607067b12, user_id=0b6a3496e6194af28e51fabdeefabd30, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:08.518709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904bf3b-6999-4f49-962f-dd68c8c862f6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:08.525068 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 815/1631] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:08.534479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84c4e3-7bb9-4b62-9649-c3fae7baed27 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:08.562241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84c4e3-7bb9-4b62-9649-c3fae7baed27 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:08.567227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 817/1632] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:08 2022] PUT /identity/v3/projects/3498c30e42ef46b884d8c2d607067b12/users/0b6a3496e6194af28e51fabdeefabd30/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:08.579939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb77ee-1ade-4b9b-8e44-c4acad6b8d22 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:08.579939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb77ee-1ade-4b9b-8e44-c4acad6b8d22 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:08.579939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb77ee-1ade-4b9b-8e44-c4acad6b8d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:08.679544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-04bb77ee-1ade-4b9b-8e44-c4acad6b8d22 None None] MFA Rules not processed for user `0b6a3496e6194af28e51fabdeefabd30`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:08.680640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-04bb77ee-1ade-4b9b-8e44-c4acad6b8d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:08.743830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e48d0-fe93-4363-8603-dc4de1c0fe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:08.745009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e48d0-fe93-4363-8603-dc4de1c0fe77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:08.745009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e48d0-fe93-4363-8603-dc4de1c0fe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:08.745221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e48d0-fe93-4363-8603-dc4de1c0fe77 admin admin] PATH_INFO: `/v3/projects/df73b49fde7a48108a4cb6e6b36957aa/users/c7bd3cafc6af4a0980701a3327b78f56/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:08.749318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e48d0-fe93-4363-8603-dc4de1c0fe77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df73b49fde7a48108a4cb6e6b36957aa, user_id=c7bd3cafc6af4a0980701a3327b78f56, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:08.757337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e48d0-fe93-4363-8603-dc4de1c0fe77 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:08.780843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 816/1633] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:08 2022] PUT /identity/v3/projects/df73b49fde7a48108a4cb6e6b36957aa/users/c7bd3cafc6af4a0980701a3327b78f56/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:08.787099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d955f7c-c79a-4a92-802f-908ef2867fe0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:08.789173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d955f7c-c79a-4a92-802f-908ef2867fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:08.789173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d955f7c-c79a-4a92-802f-908ef2867fe0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:08.800530 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 817/1634] 10.222.0.40 () {54 vars in 912 bytes} [Mon Aug 1 07:37:08 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:37:08.801855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650865c1-faa6-49bf-bfca-84ced1fc55f6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:08.802220 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650865c1-faa6-49bf-bfca-84ced1fc55f6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:08.805840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650865c1-faa6-49bf-bfca-84ced1fc55f6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:08.807823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 818/1635] 10.222.0.40 () {54 vars in 912 bytes} [Mon Aug 1 07:37:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:37:08.809318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38544643-403c-44b6-815b-ea8fd237bf98 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:08.809605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38544643-403c-44b6-815b-ea8fd237bf98 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:08.809885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38544643-403c-44b6-815b-ea8fd237bf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:08.849627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-38544643-403c-44b6-815b-ea8fd237bf98 None None] MFA Rules not processed for user `c7bd3cafc6af4a0980701a3327b78f56`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:08.850990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-38544643-403c-44b6-815b-ea8fd237bf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:08.860890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 818/1636] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:08 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:08.868673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b52a2-2581-4aeb-9a8c-58f93f1df951 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:08.868673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b52a2-2581-4aeb-9a8c-58f93f1df951 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:08.869362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b52a2-2581-4aeb-9a8c-58f93f1df951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:08.913193 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-435b52a2-2581-4aeb-9a8c-58f93f1df951 None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:08.914612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-435b52a2-2581-4aeb-9a8c-58f93f1df951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:08.961494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 819/1637] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:08 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:08.962722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e95ef-b1ea-4fdf-a853-49d3bb8ce938 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:08.963011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e95ef-b1ea-4fdf-a853-49d3bb8ce938 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:08.963332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e95ef-b1ea-4fdf-a853-49d3bb8ce938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:09.011646 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 819/1638] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:37:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:09.011871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1f0e95ef-b1ea-4fdf-a853-49d3bb8ce938 None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:09.013152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b52a2-2581-4aeb-9a8c-58f93f1df951 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:09.015835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1f0e95ef-b1ea-4fdf-a853-49d3bb8ce938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:09.028217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 820/1639] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:37:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:09.029283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e95ef-b1ea-4fdf-a853-49d3bb8ce938 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:09.037872 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:09.142552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b52a2-2581-4aeb-9a8c-58f93f1df951 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:09.149136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b9c61-abf2-492b-bb28-d62daae09ade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:09.149983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b9c61-abf2-492b-bb28-d62daae09ade service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:09.150343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b9c61-abf2-492b-bb28-d62daae09ade service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:09.150685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b9c61-abf2-492b-bb28-d62daae09ade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:09.150919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e95ef-b1ea-4fdf-a853-49d3bb8ce938 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:09.160144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896085c-ac41-4036-b3c0-63b02972db5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:09.161558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896085c-ac41-4036-b3c0-63b02972db5f service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:09.162004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896085c-ac41-4036-b3c0-63b02972db5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:09.162447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896085c-ac41-4036-b3c0-63b02972db5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:09.189826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896085c-ac41-4036-b3c0-63b02972db5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:09.191332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896085c-ac41-4036-b3c0-63b02972db5f service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:09.226920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 821/1640] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:09 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:09.265533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896085c-ac41-4036-b3c0-63b02972db5f service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:09.279341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896085c-ac41-4036-b3c0-63b02972db5f service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:09.288043 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b9c61-abf2-492b-bb28-d62daae09ade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:09.289987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b9c61-abf2-492b-bb28-d62daae09ade service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:09.290915 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cf3fe-a414-4922-8c3e-ba1ad44870a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:09.291983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cf3fe-a414-4922-8c3e-ba1ad44870a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:09.292430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cf3fe-a414-4922-8c3e-ba1ad44870a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:09.292879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cf3fe-a414-4922-8c3e-ba1ad44870a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:09.307517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 820/1641] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:09 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:09.322779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cf3fe-a414-4922-8c3e-ba1ad44870a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:09.324671 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cf3fe-a414-4922-8c3e-ba1ad44870a9 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:09.339320 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:09.359639 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 822/1642] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:37:09.424725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b9c61-abf2-492b-bb28-d62daae09ade service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:09.436195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b9c61-abf2-492b-bb28-d62daae09ade service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:09.442372 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5b6cf-7adb-4bd3-b2d4-05ef15ec48fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:09.443609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5b6cf-7adb-4bd3-b2d4-05ef15ec48fb service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:09.444633 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5b6cf-7adb-4bd3-b2d4-05ef15ec48fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:09.445117 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5b6cf-7adb-4bd3-b2d4-05ef15ec48fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:09.474322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5b6cf-7adb-4bd3-b2d4-05ef15ec48fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:09.476865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5b6cf-7adb-4bd3-b2d4-05ef15ec48fb service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:09.521509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 821/1643] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:09.776723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cf3fe-a414-4922-8c3e-ba1ad44870a9 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:09.786231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cf3fe-a414-4922-8c3e-ba1ad44870a9 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:09.798489 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5981446-54ea-4f1f-9c30-036efd9dc3ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:09.800575 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5981446-54ea-4f1f-9c30-036efd9dc3ad service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:09.802366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5981446-54ea-4f1f-9c30-036efd9dc3ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:09.802882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5981446-54ea-4f1f-9c30-036efd9dc3ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:09.968638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5981446-54ea-4f1f-9c30-036efd9dc3ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:09.976590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5981446-54ea-4f1f-9c30-036efd9dc3ad service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:09.999973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 823/1644] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:09 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:10.676544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5b6cf-7adb-4bd3-b2d4-05ef15ec48fb service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:10.773111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5b6cf-7adb-4bd3-b2d4-05ef15ec48fb service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:10.778757 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:10.779596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:10.780048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:10.780429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] PATH_INFO: `/v3/users/f38a13230b0e4baab9148a2e20bbe73b` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:10.781808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38a13230b0e4baab9148a2e20bbe73b)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:10.782996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: WARNING py.warnings [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:37:10.782996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: warnings.warn(msg) Aug 01 07:37:10.782996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: Aug 01 07:37:10.784555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:10.953720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:10.984108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38a13230b0e4baab9148a2e20bbe73b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:11.033902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38a13230b0e4baab9148a2e20bbe73b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:11.035003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38a13230b0e4baab9148a2e20bbe73b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:11.057715 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38a13230b0e4baab9148a2e20bbe73b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:11.070711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38a13230b0e4baab9148a2e20bbe73b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:11.081415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 822/1645] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:10 2022] DELETE /identity/v3/users/f38a13230b0e4baab9148a2e20bbe73b => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:11.086594 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b36dc-e1e6-4310-9127-65ff04e60273 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:11.087089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b36dc-e1e6-4310-9127-65ff04e60273 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:11.087178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b36dc-e1e6-4310-9127-65ff04e60273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:11.130678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cb1b36dc-e1e6-4310-9127-65ff04e60273 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:11.135019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb1b36dc-e1e6-4310-9127-65ff04e60273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:11.274976 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 824/1646] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:11.299246 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:11.300888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:11.497034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b36dc-e1e6-4310-9127-65ff04e60273 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:11.498788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb1b36dc-e1e6-4310-9127-65ff04e60273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:11.517099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b36dc-e1e6-4310-9127-65ff04e60273 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:11.535302 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:11.536343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:11.536911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:11.537391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] PATH_INFO: `/v3/users/2b52798ad058487192937fe12177cf70` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:11.538942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b52798ad058487192937fe12177cf70)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:11.539155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fb006-d8f7-481d-862c-28e994579a3e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:11.541565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:11.545467 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45a762-a468-4e6c-8ceb-d4758d0a89a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:11.546111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45a762-a468-4e6c-8ceb-d4758d0a89a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:11.546624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45a762-a468-4e6c-8ceb-d4758d0a89a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:11.547119 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45a762-a468-4e6c-8ceb-d4758d0a89a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:11.571248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45a762-a468-4e6c-8ceb-d4758d0a89a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:11.572797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45a762-a468-4e6c-8ceb-d4758d0a89a6 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:11.613443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 823/1647] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:11.636710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b52798ad058487192937fe12177cf70'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:11.648652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b52798ad058487192937fe12177cf70'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:11.676126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b52798ad058487192937fe12177cf70'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:11.678905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b52798ad058487192937fe12177cf70'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:11.681504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b52798ad058487192937fe12177cf70'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:11.730173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 825/1648] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:11 2022] DELETE /identity/v3/users/2b52798ad058487192937fe12177cf70 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:11.734724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617b61b-df6d-47cf-87f7-9642f81312a3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:11.735267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617b61b-df6d-47cf-87f7-9642f81312a3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:11.735355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617b61b-df6d-47cf-87f7-9642f81312a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:11.769349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2617b61b-df6d-47cf-87f7-9642f81312a3 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:11.770594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2617b61b-df6d-47cf-87f7-9642f81312a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:11.913980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 824/1649] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:11.987328 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:11.993222 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:12.060483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:12.177993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928ce7b-9869-4ee9-8277-b5f170a04b56 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:12.192372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3325c8-238e-4371-86ec-9c2a531244c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:12.192992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3325c8-238e-4371-86ec-9c2a531244c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:12.193325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3325c8-238e-4371-86ec-9c2a531244c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:12.193604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3325c8-238e-4371-86ec-9c2a531244c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:12.217896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3325c8-238e-4371-86ec-9c2a531244c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:12.221958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3325c8-238e-4371-86ec-9c2a531244c1 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:12.280584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 826/1650] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:13.722221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617b61b-df6d-47cf-87f7-9642f81312a3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:13.723722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2617b61b-df6d-47cf-87f7-9642f81312a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:13.739841 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3325c8-238e-4371-86ec-9c2a531244c1 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:13.742216 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dd3325c8-238e-4371-86ec-9c2a531244c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:13.780273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617b61b-df6d-47cf-87f7-9642f81312a3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:13.796297 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3325c8-238e-4371-86ec-9c2a531244c1 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:13.801608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:13.805192 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:13.806604 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:13.808559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5 admin admin] PATH_INFO: `/v3/projects/2f0667d27d174403b879f4c13a318fa3` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:13.814890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f0667d27d174403b879f4c13a318fa3)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:13.822980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:13.826196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:13.827032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:13.828079 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:13.828510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] PATH_INFO: `/v3/projects/640f07cfa0a543c8a1a4c3d0d74ceebd` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:13.830064 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] RBAC: Authorizing `identity:delete_project(project_id=640f07cfa0a543c8a1a4c3d0d74ceebd)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:13.838399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 07:37:13.838399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Aug 01 07:37:13.838399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: Aug 01 07:37:13.841803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:13.949650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f0667d27d174403b879f4c13a318fa3', 'request_id': 'req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:13.962320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:13.966497 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f0667d27d174403b879f4c13a318fa3', 'request_id': 'req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:13.984684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f0667d27d174403b879f4c13a318fa3', 'request_id': 'req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.019263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5 admin admin] The token cache is being invalidate because project 2f0667d27d174403b879f4c13a318fa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:14.019607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.020299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-70a05b9b-7d94-48ca-9ea5-83ad65ca92c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.035827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 825/1651] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:13 2022] DELETE /identity/v3/projects/2f0667d27d174403b879f4c13a318fa3 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:14.038515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3531be7-1672-43e2-a2c6-1c108185c74e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:14.038515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3531be7-1672-43e2-a2c6-1c108185c74e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:14.038954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3531be7-1672-43e2-a2c6-1c108185c74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:14.043167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '640f07cfa0a543c8a1a4c3d0d74ceebd', 'request_id': 'req-3a04bebc-9ac2-4210-8ef1-03275d93365a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.062160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '640f07cfa0a543c8a1a4c3d0d74ceebd', 'request_id': 'req-3a04bebc-9ac2-4210-8ef1-03275d93365a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.077694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c3531be7-1672-43e2-a2c6-1c108185c74e None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:14.079104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '640f07cfa0a543c8a1a4c3d0d74ceebd', 'request_id': 'req-3a04bebc-9ac2-4210-8ef1-03275d93365a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.079336 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c3531be7-1672-43e2-a2c6-1c108185c74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:14.198073 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] The token cache is being invalidate because project 640f07cfa0a543c8a1a4c3d0d74ceebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:14.199898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.204143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.213191 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 827/1652] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:13 2022] DELETE /identity/v3/projects/640f07cfa0a543c8a1a4c3d0d74ceebd => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:14.217959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:14.227208 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:14.245732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 826/1653] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:14.274093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac5f22-4fc0-4f93-b229-7893b1bec26b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:14.274403 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac5f22-4fc0-4f93-b229-7893b1bec26b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:14.274677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac5f22-4fc0-4f93-b229-7893b1bec26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:14.294792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04bebc-9ac2-4210-8ef1-03275d93365a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:14.319405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:14.320663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:14.321165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:14.321688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] PATH_INFO: `/v3/users/0b6a3496e6194af28e51fabdeefabd30` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:14.327369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6a3496e6194af28e51fabdeefabd30)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:14.330675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:14.340857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f8ac5f22-4fc0-4f93-b229-7893b1bec26b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:14.342615 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f8ac5f22-4fc0-4f93-b229-7893b1bec26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:14.370421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 827/1654] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:14.374663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac5f22-4fc0-4f93-b229-7893b1bec26b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:14.393513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac5f22-4fc0-4f93-b229-7893b1bec26b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:14.407295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e1cff-525e-4671-95f5-6707e156d641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:14.408229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e1cff-525e-4671-95f5-6707e156d641 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:14.408531 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e1cff-525e-4671-95f5-6707e156d641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:14.408814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e1cff-525e-4671-95f5-6707e156d641 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:14.409549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e1cff-525e-4671-95f5-6707e156d641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:14.410008 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e1cff-525e-4671-95f5-6707e156d641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:14.411765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e1cff-525e-4671-95f5-6707e156d641 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:14.440539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 828/1655] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:14.446383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e1cff-525e-4671-95f5-6707e156d641 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:14.465390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e1cff-525e-4671-95f5-6707e156d641 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:14.471539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6a3496e6194af28e51fabdeefabd30'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.479468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df76910-2c10-40b5-8189-11030d6c5005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:14.480273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df76910-2c10-40b5-8189-11030d6c5005 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:14.480572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df76910-2c10-40b5-8189-11030d6c5005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:14.480908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df76910-2c10-40b5-8189-11030d6c5005 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:14.481525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df76910-2c10-40b5-8189-11030d6c5005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:14.483508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df76910-2c10-40b5-8189-11030d6c5005 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:14.488576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6a3496e6194af28e51fabdeefabd30'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.508407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6a3496e6194af28e51fabdeefabd30'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.509274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6a3496e6194af28e51fabdeefabd30'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.509998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6a3496e6194af28e51fabdeefabd30'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:14.543559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 828/1656] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:14 2022] DELETE /identity/v3/users/0b6a3496e6194af28e51fabdeefabd30 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:14.561218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 829/1657] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:14 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:37:14.565492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:14.567173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:14.583218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:14.736979 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fd82b-f682-4038-9986-270098f71598 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:14.838896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b3b8a-f771-45b2-8ee9-adb4caacaa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:14.844958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b3b8a-f771-45b2-8ee9-adb4caacaa57 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:14.846365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b3b8a-f771-45b2-8ee9-adb4caacaa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:14.846943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b3b8a-f771-45b2-8ee9-adb4caacaa57 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:14.848281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b3b8a-f771-45b2-8ee9-adb4caacaa57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:14.852041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b3b8a-f771-45b2-8ee9-adb4caacaa57 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:14.894593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df76910-2c10-40b5-8189-11030d6c5005 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:14.896399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 829/1658] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:14 2022] POST /identity/v3/users => generated 288 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:14.915111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df76910-2c10-40b5-8189-11030d6c5005 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:14.935525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cecef3-7b92-4d30-bf21-2ead35237394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:14.936256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cecef3-7b92-4d30-bf21-2ead35237394 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:14.936643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cecef3-7b92-4d30-bf21-2ead35237394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:14.937031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cecef3-7b92-4d30-bf21-2ead35237394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:14.937825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cecef3-7b92-4d30-bf21-2ead35237394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:14.942737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cecef3-7b92-4d30-bf21-2ead35237394 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:14.995552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b3b8a-f771-45b2-8ee9-adb4caacaa57 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:14.999706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 830/1659] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:15.017795 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b3b8a-f771-45b2-8ee9-adb4caacaa57 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:15.023518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a41d0-3bd2-4778-8ea6-587b6a0de6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:15.024589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a41d0-3bd2-4778-8ea6-587b6a0de6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:15.025132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a41d0-3bd2-4778-8ea6-587b6a0de6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:15.025637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a41d0-3bd2-4778-8ea6-587b6a0de6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:15.026735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a41d0-3bd2-4778-8ea6-587b6a0de6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:15.028885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a41d0-3bd2-4778-8ea6-587b6a0de6f7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:15.045794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 830/1660] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:15.046681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cecef3-7b92-4d30-bf21-2ead35237394 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:15.061655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cecef3-7b92-4d30-bf21-2ead35237394 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:15.068763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd1df3-631e-4092-897a-6f1b6aef3292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:15.070684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd1df3-631e-4092-897a-6f1b6aef3292 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:15.071314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd1df3-631e-4092-897a-6f1b6aef3292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:15.071811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd1df3-631e-4092-897a-6f1b6aef3292 admin admin] PATH_INFO: `/v3/projects/c0e0dd79b53841b1be8b1a4cedad91b1/users/5ab869e1fc0847bb87fdd64859141166/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:15.099165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd1df3-631e-4092-897a-6f1b6aef3292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0e0dd79b53841b1be8b1a4cedad91b1, user_id=5ab869e1fc0847bb87fdd64859141166, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:15.104756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd1df3-631e-4092-897a-6f1b6aef3292 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:15.133921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 831/1661] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:15 2022] PUT /identity/v3/projects/c0e0dd79b53841b1be8b1a4cedad91b1/users/5ab869e1fc0847bb87fdd64859141166/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:37:15.149132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbe8d0-9012-4b25-b1a9-bce7d518d8cd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:15.149417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbe8d0-9012-4b25-b1a9-bce7d518d8cd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:15.149864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbe8d0-9012-4b25-b1a9-bce7d518d8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:15.211572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-37dbe8d0-9012-4b25-b1a9-bce7d518d8cd None None] MFA Rules not processed for user `5ab869e1fc0847bb87fdd64859141166`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:15.212940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-37dbe8d0-9012-4b25-b1a9-bce7d518d8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:15.379944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 831/1662] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:15.386440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd1df3-631e-4092-897a-6f1b6aef3292 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:15.514367 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd1df3-631e-4092-897a-6f1b6aef3292 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:15.529537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4d02e-a43f-46b1-8d00-fe92d9492f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:15.530719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4d02e-a43f-46b1-8d00-fe92d9492f08 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:15.533043 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4d02e-a43f-46b1-8d00-fe92d9492f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:15.534187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4d02e-a43f-46b1-8d00-fe92d9492f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:15.535296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4d02e-a43f-46b1-8d00-fe92d9492f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:15.540755 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4d02e-a43f-46b1-8d00-fe92d9492f08 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:15.570816 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:15.625957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbe8d0-9012-4b25-b1a9-bce7d518d8cd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:15.626180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 832/1663] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:15 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:15.643047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbe8d0-9012-4b25-b1a9-bce7d518d8cd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:15.715445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:15.798808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68648a-f645-4cdc-baae-0b058634dacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:15.799793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68648a-f645-4cdc-baae-0b058634dacc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:15.800361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68648a-f645-4cdc-baae-0b058634dacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:15.800843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68648a-f645-4cdc-baae-0b058634dacc admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:15.801721 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68648a-f645-4cdc-baae-0b058634dacc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:15.804876 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68648a-f645-4cdc-baae-0b058634dacc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:15.859213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4d02e-a43f-46b1-8d00-fe92d9492f08 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:15.868836 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 832/1664] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:15 2022] POST /identity/v3/users => generated 295 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:37:15.874753 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4d02e-a43f-46b1-8d00-fe92d9492f08 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:15.882812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ff330-d8b6-4309-a5af-0a4f54fec215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:15.883778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ff330-d8b6-4309-a5af-0a4f54fec215 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:15.884431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ff330-d8b6-4309-a5af-0a4f54fec215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:15.884918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ff330-d8b6-4309-a5af-0a4f54fec215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:15.885968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ff330-d8b6-4309-a5af-0a4f54fec215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:15.888114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ff330-d8b6-4309-a5af-0a4f54fec215 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:15.916885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68648a-f645-4cdc-baae-0b058634dacc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:15.917367 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 833/1665] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:15.933943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68648a-f645-4cdc-baae-0b058634dacc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:15.954424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44476a07-b5e3-467f-b058-880202e58819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:15.963291 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44476a07-b5e3-467f-b058-880202e58819 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:15.963588 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44476a07-b5e3-467f-b058-880202e58819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:15.963873 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44476a07-b5e3-467f-b058-880202e58819 admin admin] PATH_INFO: `/v3/projects/7fefa59fb83944babc1c24b48d3bcd93/users/f546368db1d4491eb95dd5b08b13343c/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:15.998479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44476a07-b5e3-467f-b058-880202e58819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fefa59fb83944babc1c24b48d3bcd93, user_id=f546368db1d4491eb95dd5b08b13343c, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:16.002224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44476a07-b5e3-467f-b058-880202e58819 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:16.034881 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 833/1666] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:15 2022] PUT /identity/v3/projects/7fefa59fb83944babc1c24b48d3bcd93/users/f546368db1d4491eb95dd5b08b13343c/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:37:16.035953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ff330-d8b6-4309-a5af-0a4f54fec215 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:16.051493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ff330-d8b6-4309-a5af-0a4f54fec215 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:16.215473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f306c-93ad-45b7-8fbb-cc22e1ab37cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:16.216138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f306c-93ad-45b7-8fbb-cc22e1ab37cb admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:16.216428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f306c-93ad-45b7-8fbb-cc22e1ab37cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:16.216710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f306c-93ad-45b7-8fbb-cc22e1ab37cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:16.218678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f306c-93ad-45b7-8fbb-cc22e1ab37cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:16.221592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f306c-93ad-45b7-8fbb-cc22e1ab37cb admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:16.245667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 834/1667] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:16.248797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44476a07-b5e3-467f-b058-880202e58819 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:16.268534 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b669f6-d852-4953-8cf9-e042235150c2 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:16.269013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b669f6-d852-4953-8cf9-e042235150c2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:16.269558 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b669f6-d852-4953-8cf9-e042235150c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:16.290952 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44476a07-b5e3-467f-b058-880202e58819 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:16.301116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d31225-c9a4-4cdf-a8e3-0966a4a42812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:16.301812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d31225-c9a4-4cdf-a8e3-0966a4a42812 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:16.302104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d31225-c9a4-4cdf-a8e3-0966a4a42812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:16.302388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d31225-c9a4-4cdf-a8e3-0966a4a42812 admin admin] PATH_INFO: `/v3/projects/7fefa59fb83944babc1c24b48d3bcd93/users/f546368db1d4491eb95dd5b08b13343c/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:16.306450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d31225-c9a4-4cdf-a8e3-0966a4a42812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fefa59fb83944babc1c24b48d3bcd93, user_id=f546368db1d4491eb95dd5b08b13343c, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:16.311815 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d31225-c9a4-4cdf-a8e3-0966a4a42812 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:16.325745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-40b669f6-d852-4953-8cf9-e042235150c2 None None] MFA Rules not processed for user `5958b9f3a2c4424db419e7b1c667ca92`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:16.327286 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-40b669f6-d852-4953-8cf9-e042235150c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:16.343601 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 834/1668] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:16 2022] PUT /identity/v3/projects/7fefa59fb83944babc1c24b48d3bcd93/users/f546368db1d4491eb95dd5b08b13343c/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:16.345831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d31225-c9a4-4cdf-a8e3-0966a4a42812 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:16.353041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-88d31225-c9a4-4cdf-a8e3-0966a4a42812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:16.532632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 835/1669] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:16.543273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb29e3f-4f8b-4452-99be-30767eacdaff None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:16.543273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb29e3f-4f8b-4452-99be-30767eacdaff None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:16.543911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb29e3f-4f8b-4452-99be-30767eacdaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:16.554056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d31225-c9a4-4cdf-a8e3-0966a4a42812 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:16.577235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:16.579409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:16.579947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:16.580425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] PATH_INFO: `/v3/projects/3498c30e42ef46b884d8c2d607067b12` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:16.584423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3498c30e42ef46b884d8c2d607067b12)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:16.591015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:16.604770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ceb29e3f-4f8b-4452-99be-30767eacdaff None None] MFA Rules not processed for user `f546368db1d4491eb95dd5b08b13343c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:16.606508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ceb29e3f-4f8b-4452-99be-30767eacdaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:16.747041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:16.771471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 836/1670] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:16.776281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb29e3f-4f8b-4452-99be-30767eacdaff None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:16.778871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ceb29e3f-4f8b-4452-99be-30767eacdaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:16.786448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3498c30e42ef46b884d8c2d607067b12', 'request_id': 'req-0931be92-6037-4afb-bbfe-d8183abe9d1b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:16.806409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3498c30e42ef46b884d8c2d607067b12', 'request_id': 'req-0931be92-6037-4afb-bbfe-d8183abe9d1b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:16.823224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3498c30e42ef46b884d8c2d607067b12', 'request_id': 'req-0931be92-6037-4afb-bbfe-d8183abe9d1b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:16.889009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] The token cache is being invalidate because project 3498c30e42ef46b884d8c2d607067b12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:16.891947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:16.894869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:16.902745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 835/1671] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:16 2022] DELETE /identity/v3/projects/3498c30e42ef46b884d8c2d607067b12 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:16.904810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:16.908613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:16.914514 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb29e3f-4f8b-4452-99be-30767eacdaff None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:16.932397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:17.045779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd822b7d-01c5-4d99-9df7-0fec320ed29b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:17.046432 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd822b7d-01c5-4d99-9df7-0fec320ed29b service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:17.046719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd822b7d-01c5-4d99-9df7-0fec320ed29b service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:17.046996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd822b7d-01c5-4d99-9df7-0fec320ed29b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:17.051166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bd822b7d-01c5-4d99-9df7-0fec320ed29b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:17.116321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931be92-6037-4afb-bbfe-d8183abe9d1b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:17.129890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c4b9e-e6a7-40c1-ba32-12ac3f0357de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:17.131656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c4b9e-e6a7-40c1-ba32-12ac3f0357de service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:17.132478 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c4b9e-e6a7-40c1-ba32-12ac3f0357de service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:17.139717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c4b9e-e6a7-40c1-ba32-12ac3f0357de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:17.142080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9c2c4b9e-e6a7-40c1-ba32-12ac3f0357de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:17.207731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd822b7d-01c5-4d99-9df7-0fec320ed29b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:17.211634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd822b7d-01c5-4d99-9df7-0fec320ed29b service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:17.239670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 837/1672] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:17.241660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19ff6b-ab8e-495c-80dc-64c656175a6a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:17.241946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19ff6b-ab8e-495c-80dc-64c656175a6a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:17.242282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19ff6b-ab8e-495c-80dc-64c656175a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:17.301620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5e19ff6b-ab8e-495c-80dc-64c656175a6a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:17.302940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5e19ff6b-ab8e-495c-80dc-64c656175a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:17.329403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c4b9e-e6a7-40c1-ba32-12ac3f0357de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:17.331378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c4b9e-e6a7-40c1-ba32-12ac3f0357de service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:17.358605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 836/1673] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:17.388838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:17.478053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 838/1674] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:17.486944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245974f-d433-4c80-9b7a-4e15c173bbcf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:17.487299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245974f-d433-4c80-9b7a-4e15c173bbcf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:17.487592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245974f-d433-4c80-9b7a-4e15c173bbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:17.515464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3245974f-d433-4c80-9b7a-4e15c173bbcf None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:17.517447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3245974f-d433-4c80-9b7a-4e15c173bbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:17.550991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 837/1675] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:17.552444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19ff6b-ab8e-495c-80dc-64c656175a6a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:17.565412 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19ff6b-ab8e-495c-80dc-64c656175a6a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:17.580573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9f6d2-b436-48e0-bec5-2fda35a20a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:17.581529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9f6d2-b436-48e0-bec5-2fda35a20a2f admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:17.582175 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9f6d2-b436-48e0-bec5-2fda35a20a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:17.582633 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9f6d2-b436-48e0-bec5-2fda35a20a2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:17.584001 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9f6d2-b436-48e0-bec5-2fda35a20a2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:17.585237 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9f6d2-b436-48e0-bec5-2fda35a20a2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:17.588498 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9f6d2-b436-48e0-bec5-2fda35a20a2f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:17.624126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245974f-d433-4c80-9b7a-4e15c173bbcf None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:17.631820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 839/1676] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:17.638481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245974f-d433-4c80-9b7a-4e15c173bbcf None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:17.651131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb54ea-2a91-4fd8-9d84-ebc87add0562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:17.651800 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb54ea-2a91-4fd8-9d84-ebc87add0562 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:17.652089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb54ea-2a91-4fd8-9d84-ebc87add0562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:17.652374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb54ea-2a91-4fd8-9d84-ebc87add0562 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:17.654130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb54ea-2a91-4fd8-9d84-ebc87add0562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:17.658994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb54ea-2a91-4fd8-9d84-ebc87add0562 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:17.776796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 838/1677] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:17 2022] POST /identity/v3/projects => generated 363 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:17.778439 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9f6d2-b436-48e0-bec5-2fda35a20a2f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:17.793298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9f6d2-b436-48e0-bec5-2fda35a20a2f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:17.954197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f70272-118e-423e-9949-c41976905333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:17.955160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f70272-118e-423e-9949-c41976905333 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:17.955671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f70272-118e-423e-9949-c41976905333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:17.956152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f70272-118e-423e-9949-c41976905333 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:17.957187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f70272-118e-423e-9949-c41976905333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:17.965610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f70272-118e-423e-9949-c41976905333 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:18.018968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 840/1678] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:17 2022] POST /identity/v3/users => generated 282 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:18.019953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb54ea-2a91-4fd8-9d84-ebc87add0562 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:18.041696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb54ea-2a91-4fd8-9d84-ebc87add0562 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:18.065758 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6add083-581e-4d51-bb36-8ac1ef62f6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:18.066199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6add083-581e-4d51-bb36-8ac1ef62f6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:18.066490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6add083-581e-4d51-bb36-8ac1ef62f6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:18.074140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6add083-581e-4d51-bb36-8ac1ef62f6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:18.076920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6add083-581e-4d51-bb36-8ac1ef62f6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:18.080308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6add083-581e-4d51-bb36-8ac1ef62f6a5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:18.125376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 839/1679] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:37:18.126775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f70272-118e-423e-9949-c41976905333 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:18.155971 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f70272-118e-423e-9949-c41976905333 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:18.171604 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6377dd-148c-4922-a61c-e0a20ace193d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:18.174313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6377dd-148c-4922-a61c-e0a20ace193d admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:18.175996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6377dd-148c-4922-a61c-e0a20ace193d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:18.176610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6377dd-148c-4922-a61c-e0a20ace193d admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:18.181578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6377dd-148c-4922-a61c-e0a20ace193d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:18.185084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6377dd-148c-4922-a61c-e0a20ace193d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:18.235810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6add083-581e-4d51-bb36-8ac1ef62f6a5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:18.238258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 841/1680] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:18.247888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6add083-581e-4d51-bb36-8ac1ef62f6a5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:18.265780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d857c3-26f1-4576-aedb-17a744a1ad4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:18.266483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d857c3-26f1-4576-aedb-17a744a1ad4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:18.266773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d857c3-26f1-4576-aedb-17a744a1ad4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:18.267080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d857c3-26f1-4576-aedb-17a744a1ad4a admin admin] PATH_INFO: `/v3/projects/b21cc7e2f6e642f0895d707be28d8918/users/255ec60d60f545c190413e08a96ea03c/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:18.309079 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d857c3-26f1-4576-aedb-17a744a1ad4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21cc7e2f6e642f0895d707be28d8918, user_id=255ec60d60f545c190413e08a96ea03c, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:18.314468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d857c3-26f1-4576-aedb-17a744a1ad4a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:18.354757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 840/1681] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:18 2022] PUT /identity/v3/projects/b21cc7e2f6e642f0895d707be28d8918/users/255ec60d60f545c190413e08a96ea03c/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:18.365330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae2d80-7a6c-4f1a-8faa-e594d5526c88 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:18.365669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae2d80-7a6c-4f1a-8faa-e594d5526c88 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:18.365948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae2d80-7a6c-4f1a-8faa-e594d5526c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:18.400099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b9ae2d80-7a6c-4f1a-8faa-e594d5526c88 None None] MFA Rules not processed for user `255ec60d60f545c190413e08a96ea03c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:18.401485 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b9ae2d80-7a6c-4f1a-8faa-e594d5526c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:18.588453 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 842/1682] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:18.590006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d857c3-26f1-4576-aedb-17a744a1ad4a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:18.712267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:18.727546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d857c3-26f1-4576-aedb-17a744a1ad4a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:18.738618 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275cc34-f589-4280-b7c0-38ddcc20595b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:18.739571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275cc34-f589-4280-b7c0-38ddcc20595b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:18.739854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275cc34-f589-4280-b7c0-38ddcc20595b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:18.740357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275cc34-f589-4280-b7c0-38ddcc20595b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:18.741057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275cc34-f589-4280-b7c0-38ddcc20595b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:18.745600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275cc34-f589-4280-b7c0-38ddcc20595b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:18.808726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 841/1683] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:18 2022] POST /identity/v3/projects => generated 363 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:18.809187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae2d80-7a6c-4f1a-8faa-e594d5526c88 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:18.828542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae2d80-7a6c-4f1a-8faa-e594d5526c88 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:18.888281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:18.956427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a2e13-f87f-4505-96ba-fbbfc38c00a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:18.958027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a2e13-f87f-4505-96ba-fbbfc38c00a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:18.959751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a2e13-f87f-4505-96ba-fbbfc38c00a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:18.960346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a2e13-f87f-4505-96ba-fbbfc38c00a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:18.963153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a2e13-f87f-4505-96ba-fbbfc38c00a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:18.976970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a2e13-f87f-4505-96ba-fbbfc38c00a8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:19.030770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 843/1684] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:18 2022] POST /identity/v3/users => generated 288 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:19.032049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275cc34-f589-4280-b7c0-38ddcc20595b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:19.052638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275cc34-f589-4280-b7c0-38ddcc20595b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:19.065255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e376f7-7c16-49e1-a487-a2c62c1a79da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:19.066255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e376f7-7c16-49e1-a487-a2c62c1a79da admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:19.066724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e376f7-7c16-49e1-a487-a2c62c1a79da admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:19.067269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e376f7-7c16-49e1-a487-a2c62c1a79da admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:19.068158 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e376f7-7c16-49e1-a487-a2c62c1a79da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:19.070315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e376f7-7c16-49e1-a487-a2c62c1a79da admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:19.093962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 842/1685] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:19.094899 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a2e13-f87f-4505-96ba-fbbfc38c00a8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:19.112183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a2e13-f87f-4505-96ba-fbbfc38c00a8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:19.126447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbcf0e-fa52-414b-84eb-ab09250e2e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:19.127334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbcf0e-fa52-414b-84eb-ab09250e2e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:19.127873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbcf0e-fa52-414b-84eb-ab09250e2e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:19.128059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbcf0e-fa52-414b-84eb-ab09250e2e2a admin admin] PATH_INFO: `/v3/projects/a87b0cbceaac4a11a55a02d63737670e/users/d7df7a7188704f87912b082d37e21ae5/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:19.174760 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbcf0e-fa52-414b-84eb-ab09250e2e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87b0cbceaac4a11a55a02d63737670e, user_id=d7df7a7188704f87912b082d37e21ae5, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:19.189531 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbcf0e-fa52-414b-84eb-ab09250e2e2a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:19.227151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 844/1686] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:19 2022] PUT /identity/v3/projects/a87b0cbceaac4a11a55a02d63737670e/users/d7df7a7188704f87912b082d37e21ae5/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:19.228423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e376f7-7c16-49e1-a487-a2c62c1a79da admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:19.245673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e376f7-7c16-49e1-a487-a2c62c1a79da admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:19.375196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a0d3c-e027-49bb-82ca-c90e3e824294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:19.376212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a0d3c-e027-49bb-82ca-c90e3e824294 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:19.376872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a0d3c-e027-49bb-82ca-c90e3e824294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:19.377278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a0d3c-e027-49bb-82ca-c90e3e824294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:19.378507 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a0d3c-e027-49bb-82ca-c90e3e824294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:19.380533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a0d3c-e027-49bb-82ca-c90e3e824294 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:19.405017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 843/1687] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:19.410361 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbcf0e-fa52-414b-84eb-ab09250e2e2a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:19.431679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbcf0e-fa52-414b-84eb-ab09250e2e2a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:19.458661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9205223-46b0-4f84-aa03-0384c55032b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:19.458661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9205223-46b0-4f84-aa03-0384c55032b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:19.458661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9205223-46b0-4f84-aa03-0384c55032b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:19.458661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9205223-46b0-4f84-aa03-0384c55032b8 admin admin] PATH_INFO: `/v3/projects/a87b0cbceaac4a11a55a02d63737670e/users/d7df7a7188704f87912b082d37e21ae5/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:19.458936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9205223-46b0-4f84-aa03-0384c55032b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87b0cbceaac4a11a55a02d63737670e, user_id=d7df7a7188704f87912b082d37e21ae5, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:19.458936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9205223-46b0-4f84-aa03-0384c55032b8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:19.494722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 845/1688] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:19 2022] PUT /identity/v3/projects/a87b0cbceaac4a11a55a02d63737670e/users/d7df7a7188704f87912b082d37e21ae5/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:19.504405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839e4e6-3459-41b6-8db0-d48246cf552a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:19.504900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839e4e6-3459-41b6-8db0-d48246cf552a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:19.505578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839e4e6-3459-41b6-8db0-d48246cf552a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:19.573067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7839e4e6-3459-41b6-8db0-d48246cf552a None None] MFA Rules not processed for user `d7df7a7188704f87912b082d37e21ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:19.574531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7839e4e6-3459-41b6-8db0-d48246cf552a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:19.748263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 844/1689] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:19 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:19.791974 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9205223-46b0-4f84-aa03-0384c55032b8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:19.793750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a9205223-46b0-4f84-aa03-0384c55032b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:19.933197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9205223-46b0-4f84-aa03-0384c55032b8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:19.947909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206bcf5-a187-43d9-bd69-efdbc636baaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:19.949474 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206bcf5-a187-43d9-bd69-efdbc636baaa service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:19.949474 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206bcf5-a187-43d9-bd69-efdbc636baaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:19.950536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206bcf5-a187-43d9-bd69-efdbc636baaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:20.006007 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839e4e6-3459-41b6-8db0-d48246cf552a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:20.008982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7839e4e6-3459-41b6-8db0-d48246cf552a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:20.014296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206bcf5-a187-43d9-bd69-efdbc636baaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:20.015822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206bcf5-a187-43d9-bd69-efdbc636baaa service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:20.064472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 846/1690] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:19 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:20.142931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:20.195398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839e4e6-3459-41b6-8db0-d48246cf552a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:20.224347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d80f7d-cd77-47fe-87b8-e3dbd4e84712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:20.225130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d80f7d-cd77-47fe-87b8-e3dbd4e84712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:20.225638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d80f7d-cd77-47fe-87b8-e3dbd4e84712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:20.226137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d80f7d-cd77-47fe-87b8-e3dbd4e84712 admin admin] PATH_INFO: `/v3/users/e4a248d79a3f4aa2b7937e566508e559` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:20.227564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d80f7d-cd77-47fe-87b8-e3dbd4e84712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4a248d79a3f4aa2b7937e566508e559)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:20.229524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d80f7d-cd77-47fe-87b8-e3dbd4e84712 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:20.292092 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206bcf5-a187-43d9-bd69-efdbc636baaa service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:20.306599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206bcf5-a187-43d9-bd69-efdbc636baaa service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:20.312842 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa7a98-0f3b-449b-ae83-f6135f14b95d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:20.315600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa7a98-0f3b-449b-ae83-f6135f14b95d service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:20.316045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa7a98-0f3b-449b-ae83-f6135f14b95d service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:20.317184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa7a98-0f3b-449b-ae83-f6135f14b95d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:20.389334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2d80f7d-cd77-47fe-87b8-e3dbd4e84712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4a248d79a3f4aa2b7937e566508e559'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:20.407385 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2d80f7d-cd77-47fe-87b8-e3dbd4e84712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4a248d79a3f4aa2b7937e566508e559'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:20.440749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2d80f7d-cd77-47fe-87b8-e3dbd4e84712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4a248d79a3f4aa2b7937e566508e559'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:20.441656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2d80f7d-cd77-47fe-87b8-e3dbd4e84712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4a248d79a3f4aa2b7937e566508e559'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:20.442447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2d80f7d-cd77-47fe-87b8-e3dbd4e84712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4a248d79a3f4aa2b7937e566508e559'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:20.479748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 845/1691] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:20 2022] DELETE /identity/v3/users/e4a248d79a3f4aa2b7937e566508e559 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:20.481327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290895d-2509-4fc3-a0e9-980306a75988 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:20.481622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290895d-2509-4fc3-a0e9-980306a75988 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:20.481998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290895d-2509-4fc3-a0e9-980306a75988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:20.508943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa7a98-0f3b-449b-ae83-f6135f14b95d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:20.514162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa7a98-0f3b-449b-ae83-f6135f14b95d service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:20.526895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c290895d-2509-4fc3-a0e9-980306a75988 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:20.529745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c290895d-2509-4fc3-a0e9-980306a75988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:20.550134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 847/1692] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:20 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:20.727177 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 846/1693] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:21.020926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa7a98-0f3b-449b-ae83-f6135f14b95d service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:21.023032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-71aa7a98-0f3b-449b-ae83-f6135f14b95d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:21.095954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:21.211875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa7a98-0f3b-449b-ae83-f6135f14b95d service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:21.370154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecc576-4211-497f-96b7-2d3f80a046ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:21.370981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecc576-4211-497f-96b7-2d3f80a046ca service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:21.371801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecc576-4211-497f-96b7-2d3f80a046ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:21.372453 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecc576-4211-497f-96b7-2d3f80a046ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:21.400991 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecc576-4211-497f-96b7-2d3f80a046ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:21.415697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecc576-4211-497f-96b7-2d3f80a046ca service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:21.441557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 848/1694] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:22.048624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cc11c-3967-4795-bc06-358badad38df None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:22.049060 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cc11c-3967-4795-bc06-358badad38df None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:22.049182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cc11c-3967-4795-bc06-358badad38df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:22.212194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f20cc11c-3967-4795-bc06-358badad38df None None] MFA Rules not processed for user `db6ac4cf9dd84e338f944b4905dcdb50`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:22.213477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f20cc11c-3967-4795-bc06-358badad38df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:22.425865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:22.436575 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 847/1695] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:22.447419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae04c-e8bb-4775-b3ac-936e66c26404 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:22.447973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae04c-e8bb-4775-b3ac-936e66c26404 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:22.448066 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae04c-e8bb-4775-b3ac-936e66c26404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:22.506738 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ff6ae04c-e8bb-4775-b3ac-936e66c26404 None None] MFA Rules not processed for user `db6ac4cf9dd84e338f944b4905dcdb50`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:22.508225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ff6ae04c-e8bb-4775-b3ac-936e66c26404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:22.525653 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 849/1696] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:22.538890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cc11c-3967-4795-bc06-358badad38df None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:22.541012 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f20cc11c-3967-4795-bc06-358badad38df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:22.698899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cc11c-3967-4795-bc06-358badad38df None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:22.710717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e55f57-c87a-4af3-855f-74e64a244885 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:22.711597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e55f57-c87a-4af3-855f-74e64a244885 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:22.711983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e55f57-c87a-4af3-855f-74e64a244885 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:22.712331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e55f57-c87a-4af3-855f-74e64a244885 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:22.730804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e55f57-c87a-4af3-855f-74e64a244885 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:22.734028 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e55f57-c87a-4af3-855f-74e64a244885 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:22.764706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 848/1697] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:37:23.267457 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae04c-e8bb-4775-b3ac-936e66c26404 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:23.269121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ff6ae04c-e8bb-4775-b3ac-936e66c26404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:23.369206 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae04c-e8bb-4775-b3ac-936e66c26404 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:23.377292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4de57c-152b-4654-bb91-612cfba94292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:23.377951 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de57c-152b-4654-bb91-612cfba94292 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:23.378235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de57c-152b-4654-bb91-612cfba94292 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:23.378513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de57c-152b-4654-bb91-612cfba94292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:23.385708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4a4de57c-152b-4654-bb91-612cfba94292 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:23.499628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4de57c-152b-4654-bb91-612cfba94292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:23.502009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4de57c-152b-4654-bb91-612cfba94292 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:23.532529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 850/1698] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:23.703107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f62a0-4ec5-4543-be7d-c7f83ce8a96f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:23.706979 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f62a0-4ec5-4543-be7d-c7f83ce8a96f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:23.707333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f62a0-4ec5-4543-be7d-c7f83ce8a96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:23.707946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4de57c-152b-4654-bb91-612cfba94292 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:23.709468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4a4de57c-152b-4654-bb91-612cfba94292 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:23.735928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4de57c-152b-4654-bb91-612cfba94292 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:23.755055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bef0e-dd8f-4fc7-941d-46773e295693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:23.757044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bef0e-dd8f-4fc7-941d-46773e295693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:23.757542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bef0e-dd8f-4fc7-941d-46773e295693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:23.758024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bef0e-dd8f-4fc7-941d-46773e295693 admin admin] PATH_INFO: `/v3/projects/a589760b9c674009a0ec8007d1669f12` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:23.759539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bef0e-dd8f-4fc7-941d-46773e295693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a589760b9c674009a0ec8007d1669f12)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:23.761553 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bef0e-dd8f-4fc7-941d-46773e295693 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:23.772479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d28f62a0-4ec5-4543-be7d-c7f83ce8a96f None None] MFA Rules not processed for user `905e317fcf49428ea1fa708dfabc958e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:23.775094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d28f62a0-4ec5-4543-be7d-c7f83ce8a96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:23.803421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:23.891694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4f2bef0e-dd8f-4fc7-941d-46773e295693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a589760b9c674009a0ec8007d1669f12', 'request_id': 'req-4f2bef0e-dd8f-4fc7-941d-46773e295693'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:23.904099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4f2bef0e-dd8f-4fc7-941d-46773e295693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a589760b9c674009a0ec8007d1669f12', 'request_id': 'req-4f2bef0e-dd8f-4fc7-941d-46773e295693'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:23.916857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4f2bef0e-dd8f-4fc7-941d-46773e295693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a589760b9c674009a0ec8007d1669f12', 'request_id': 'req-4f2bef0e-dd8f-4fc7-941d-46773e295693'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:23.944909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4f2bef0e-dd8f-4fc7-941d-46773e295693 admin admin] The token cache is being invalidate because project a589760b9c674009a0ec8007d1669f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:23.945271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4f2bef0e-dd8f-4fc7-941d-46773e295693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:23.949824 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4f2bef0e-dd8f-4fc7-941d-46773e295693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:23.954750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 851/1699] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:23 2022] DELETE /identity/v3/projects/a589760b9c674009a0ec8007d1669f12 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:23.956123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b3a5e-e522-49d4-88f9-71027877c69c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:23.956420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b3a5e-e522-49d4-88f9-71027877c69c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:23.956705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b3a5e-e522-49d4-88f9-71027877c69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:24.015131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 849/1700] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:24.016766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-328b3a5e-e522-49d4-88f9-71027877c69c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:24.022059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-328b3a5e-e522-49d4-88f9-71027877c69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:24.025112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b14ab-6f63-401d-817b-8b20facc1ec6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:24.025406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b14ab-6f63-401d-817b-8b20facc1ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:24.025681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b14ab-6f63-401d-817b-8b20facc1ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:24.085182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-666b14ab-6f63-401d-817b-8b20facc1ec6 None None] MFA Rules not processed for user `905e317fcf49428ea1fa708dfabc958e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:24.088839 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-666b14ab-6f63-401d-817b-8b20facc1ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:24.173523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 852/1701] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:24.175568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115da7f7-2af9-4a7f-8e32-9f2bc8b0bda0 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:24.176189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115da7f7-2af9-4a7f-8e32-9f2bc8b0bda0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:24.176608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115da7f7-2af9-4a7f-8e32-9f2bc8b0bda0 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:24.178458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 853/1702] 10.222.0.40 () {54 vars in 912 bytes} [Mon Aug 1 07:37:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:37:24.184541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4669b6f-cafb-4644-a416-8d3c08a073cf None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:24.186098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4669b6f-cafb-4644-a416-8d3c08a073cf None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:24.186531 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4669b6f-cafb-4644-a416-8d3c08a073cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:24.238307 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b4669b6f-cafb-4644-a416-8d3c08a073cf None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:24.243319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b4669b6f-cafb-4644-a416-8d3c08a073cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:24.246527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 850/1703] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:24.248019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29575e-2206-4368-a12a-70c991b9ff16 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:24.248324 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29575e-2206-4368-a12a-70c991b9ff16 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:24.249644 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29575e-2206-4368-a12a-70c991b9ff16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:24.270352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 854/1704] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:24.272133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4669b6f-cafb-4644-a416-8d3c08a073cf None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:24.276856 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b4669b6f-cafb-4644-a416-8d3c08a073cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:24.361077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:24.378808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6c29575e-2206-4368-a12a-70c991b9ff16 None None] MFA Rules not processed for user `138ede2854894a3b995435980965911c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:24.383984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6c29575e-2206-4368-a12a-70c991b9ff16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:24.538481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 851/1705] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:37:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:24.539748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29575e-2206-4368-a12a-70c991b9ff16 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:24.559140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29575e-2206-4368-a12a-70c991b9ff16 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:24.564631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4669b6f-cafb-4644-a416-8d3c08a073cf None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:24.570043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca974261-4a41-47d5-a62d-28a83372a89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:24.570869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca974261-4a41-47d5-a62d-28a83372a89e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:24.571549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca974261-4a41-47d5-a62d-28a83372a89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:24.572144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca974261-4a41-47d5-a62d-28a83372a89e admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:24.573572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca974261-4a41-47d5-a62d-28a83372a89e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:24.574534 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca974261-4a41-47d5-a62d-28a83372a89e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:24.577253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca974261-4a41-47d5-a62d-28a83372a89e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:24.579271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f987b-ba2a-498b-aaac-22c053a8e096 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:24.580213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f987b-ba2a-498b-aaac-22c053a8e096 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:24.580702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f987b-ba2a-498b-aaac-22c053a8e096 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:24.581178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f987b-ba2a-498b-aaac-22c053a8e096 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:24.600991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 852/1706] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:24.602196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca974261-4a41-47d5-a62d-28a83372a89e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:24.603811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f987b-ba2a-498b-aaac-22c053a8e096 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:24.605671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f987b-ba2a-498b-aaac-22c053a8e096 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:24.606425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ca974261-4a41-47d5-a62d-28a83372a89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:24.625943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 855/1707] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:24.627178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f987b-ba2a-498b-aaac-22c053a8e096 service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:24.627580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca974261-4a41-47d5-a62d-28a83372a89e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:24.636522 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b42d5-d804-4510-9539-e31f14cf12d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:24.637245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b42d5-d804-4510-9539-e31f14cf12d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:24.637740 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b42d5-d804-4510-9539-e31f14cf12d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:24.638204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b42d5-d804-4510-9539-e31f14cf12d9 admin admin] PATH_INFO: `/v3/users/255ec60d60f545c190413e08a96ea03c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:24.639545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b42d5-d804-4510-9539-e31f14cf12d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=255ec60d60f545c190413e08a96ea03c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:24.642007 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b42d5-d804-4510-9539-e31f14cf12d9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:24.657504 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f987b-ba2a-498b-aaac-22c053a8e096 service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:24.676315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1772c6f-0a0d-4fc6-97eb-5294365a0911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:24.677201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1772c6f-0a0d-4fc6-97eb-5294365a0911 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:24.677714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1772c6f-0a0d-4fc6-97eb-5294365a0911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:24.678095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1772c6f-0a0d-4fc6-97eb-5294365a0911 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:24.680299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1772c6f-0a0d-4fc6-97eb-5294365a0911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:24.682462 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1772c6f-0a0d-4fc6-97eb-5294365a0911 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:24.711672 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:24.726108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7c3b42d5-d804-4510-9539-e31f14cf12d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '255ec60d60f545c190413e08a96ea03c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:24.733853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 856/1708] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:24 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:24.735145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1772c6f-0a0d-4fc6-97eb-5294365a0911 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:24.753796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7c3b42d5-d804-4510-9539-e31f14cf12d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '255ec60d60f545c190413e08a96ea03c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:24.769911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7c3b42d5-d804-4510-9539-e31f14cf12d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '255ec60d60f545c190413e08a96ea03c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:24.770735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7c3b42d5-d804-4510-9539-e31f14cf12d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '255ec60d60f545c190413e08a96ea03c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:24.772926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7c3b42d5-d804-4510-9539-e31f14cf12d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '255ec60d60f545c190413e08a96ea03c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:24.791429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 853/1709] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:24 2022] DELETE /identity/v3/users/255ec60d60f545c190413e08a96ea03c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:24.793043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a37f9-d774-464c-a771-d70e4fef0b54 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:24.793380 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a37f9-d774-464c-a771-d70e4fef0b54 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:24.793641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a37f9-d774-464c-a771-d70e4fef0b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:24.832615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-594a37f9-d774-464c-a771-d70e4fef0b54 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:24.834971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-594a37f9-d774-464c-a771-d70e4fef0b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:24.995436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1772c6f-0a0d-4fc6-97eb-5294365a0911 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:25.056507 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 854/1710] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:25.058711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a37f9-d774-464c-a771-d70e4fef0b54 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:25.063717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-594a37f9-d774-464c-a771-d70e4fef0b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:25.111041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523ccc7-3ccc-4074-95fd-49341d017c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:25.111717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523ccc7-3ccc-4074-95fd-49341d017c74 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:25.111988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523ccc7-3ccc-4074-95fd-49341d017c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:25.112266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523ccc7-3ccc-4074-95fd-49341d017c74 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:25.113416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:25.113948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523ccc7-3ccc-4074-95fd-49341d017c74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:25.116050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523ccc7-3ccc-4074-95fd-49341d017c74 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:25.149990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 857/1711] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:24 2022] POST /identity/v3/users => generated 286 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:25.154498 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523ccc7-3ccc-4074-95fd-49341d017c74 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:25.157077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1523ccc7-3ccc-4074-95fd-49341d017c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:25.181031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a37f9-d774-464c-a771-d70e4fef0b54 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:25.190614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523ccc7-3ccc-4074-95fd-49341d017c74 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:25.200548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da5ebd-cef7-4770-96ce-1e40a34595d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:25.201315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da5ebd-cef7-4770-96ce-1e40a34595d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:25.201621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da5ebd-cef7-4770-96ce-1e40a34595d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:25.201879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da5ebd-cef7-4770-96ce-1e40a34595d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:25.204469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f7da5ebd-cef7-4770-96ce-1e40a34595d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:25.205234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3940afa-cd24-4553-a9c1-895b5caf2fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:25.210889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3940afa-cd24-4553-a9c1-895b5caf2fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:25.210889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3940afa-cd24-4553-a9c1-895b5caf2fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:25.210889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3940afa-cd24-4553-a9c1-895b5caf2fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:25.212341 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3940afa-cd24-4553-a9c1-895b5caf2fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:25.215777 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3940afa-cd24-4553-a9c1-895b5caf2fd1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:25.216611 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da5ebd-cef7-4770-96ce-1e40a34595d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:25.218211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da5ebd-cef7-4770-96ce-1e40a34595d8 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:25.233366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 855/1712] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:25.234248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da5ebd-cef7-4770-96ce-1e40a34595d8 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:25.245621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da5ebd-cef7-4770-96ce-1e40a34595d8 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:25.249670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ca732-43c4-47a9-b8bb-eb2f3553f1df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:25.250072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ca732-43c4-47a9-b8bb-eb2f3553f1df service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:25.250362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ca732-43c4-47a9-b8bb-eb2f3553f1df service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:25.250635 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ca732-43c4-47a9-b8bb-eb2f3553f1df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:25.256595 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 858/1713] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:25.266930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3940afa-cd24-4553-a9c1-895b5caf2fd1 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:25.276057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ca732-43c4-47a9-b8bb-eb2f3553f1df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:25.277784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ca732-43c4-47a9-b8bb-eb2f3553f1df service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:25.288232 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3940afa-cd24-4553-a9c1-895b5caf2fd1 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:25.300824 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b33d3e-810b-442e-9a1c-a3c398034b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:25.301594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b33d3e-810b-442e-9a1c-a3c398034b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:25.301763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b33d3e-810b-442e-9a1c-a3c398034b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:25.302026 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b33d3e-810b-442e-9a1c-a3c398034b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:25.303391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b33d3e-810b-442e-9a1c-a3c398034b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:25.305090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b33d3e-810b-442e-9a1c-a3c398034b4e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:25.308421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 856/1714] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:25.324180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 859/1715] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:25.326572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ca732-43c4-47a9-b8bb-eb2f3553f1df service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:25.347826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ca732-43c4-47a9-b8bb-eb2f3553f1df service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:25.368550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52baffdf-6484-4cd1-9a95-d138851b4c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:25.369269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52baffdf-6484-4cd1-9a95-d138851b4c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:25.369557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52baffdf-6484-4cd1-9a95-d138851b4c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:25.369837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52baffdf-6484-4cd1-9a95-d138851b4c92 admin admin] PATH_INFO: `/v3/projects/b32dac66ce054e938e3d060e8a0f10a2/users/e7fcfa754e5e4123a1515d487835496f/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:25.406801 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52baffdf-6484-4cd1-9a95-d138851b4c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32dac66ce054e938e3d060e8a0f10a2, user_id=e7fcfa754e5e4123a1515d487835496f, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:25.410544 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52baffdf-6484-4cd1-9a95-d138851b4c92 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:25.431874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 857/1716] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:25 2022] PUT /identity/v3/projects/b32dac66ce054e938e3d060e8a0f10a2/users/e7fcfa754e5e4123a1515d487835496f/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:25.445239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df196156-cacf-4b56-9202-caa125f7be25 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:25.445635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df196156-cacf-4b56-9202-caa125f7be25 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:25.446060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df196156-cacf-4b56-9202-caa125f7be25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:25.518123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-df196156-cacf-4b56-9202-caa125f7be25 None None] MFA Rules not processed for user `e7fcfa754e5e4123a1515d487835496f`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:25.519719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-df196156-cacf-4b56-9202-caa125f7be25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:25.593421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedef86-6022-4386-b968-2b052bd9570d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:25.593734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedef86-6022-4386-b968-2b052bd9570d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:25.594253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedef86-6022-4386-b968-2b052bd9570d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:25.598027 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 858/1717] 10.222.0.40 () {54 vars in 912 bytes} [Mon Aug 1 07:37:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:37:25.609242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8d723-0a81-4d7f-babd-0d8d0ba5e80f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:25.609535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8d723-0a81-4d7f-babd-0d8d0ba5e80f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:25.609821 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8d723-0a81-4d7f-babd-0d8d0ba5e80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:25.633907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 860/1718] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:25.635955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df196156-cacf-4b56-9202-caa125f7be25 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:25.657423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7ec8d723-0a81-4d7f-babd-0d8d0ba5e80f None None] MFA Rules not processed for user `138ede2854894a3b995435980965911c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:25.666790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:25.669949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7ec8d723-0a81-4d7f-babd-0d8d0ba5e80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:25.788512 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df196156-cacf-4b56-9202-caa125f7be25 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:25.799552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 859/1719] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:37:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:25.809602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a35320-31fd-4199-8119-7ff2f3577637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:25.810482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a35320-31fd-4199-8119-7ff2f3577637 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:25.810843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a35320-31fd-4199-8119-7ff2f3577637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:25.811365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a35320-31fd-4199-8119-7ff2f3577637 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:25.812134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a35320-31fd-4199-8119-7ff2f3577637 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:25.814708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a35320-31fd-4199-8119-7ff2f3577637 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:25.818039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8d723-0a81-4d7f-babd-0d8d0ba5e80f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:25.820451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7ec8d723-0a81-4d7f-babd-0d8d0ba5e80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:25.838431 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8d723-0a81-4d7f-babd-0d8d0ba5e80f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:25.850536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881c53b-f971-406b-a57c-ee56fb5d1bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:25.851475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881c53b-f971-406b-a57c-ee56fb5d1bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:25.852664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881c53b-f971-406b-a57c-ee56fb5d1bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:25.853132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881c53b-f971-406b-a57c-ee56fb5d1bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:25.865266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881c53b-f971-406b-a57c-ee56fb5d1bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:25.870217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881c53b-f971-406b-a57c-ee56fb5d1bc2 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:25.885995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 861/1720] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:25 2022] POST /identity/v3/projects => generated 371 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:25.890099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a35320-31fd-4199-8119-7ff2f3577637 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:25.911465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 860/1721] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:37:25.966168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24295298-15c5-47d4-8ecc-ecf99ce7e311 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:25.966819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24295298-15c5-47d4-8ecc-ecf99ce7e311 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:25.968169 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24295298-15c5-47d4-8ecc-ecf99ce7e311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:26.025137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-24295298-15c5-47d4-8ecc-ecf99ce7e311 None None] MFA Rules not processed for user `5958b9f3a2c4424db419e7b1c667ca92`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:26.027244 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-24295298-15c5-47d4-8ecc-ecf99ce7e311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:26.052968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a35320-31fd-4199-8119-7ff2f3577637 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:26.060840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a77c-6e58-4a95-9863-4aa33c88855d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:26.062199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a77c-6e58-4a95-9863-4aa33c88855d admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:26.062490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a77c-6e58-4a95-9863-4aa33c88855d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:26.062770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a77c-6e58-4a95-9863-4aa33c88855d admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:26.063479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a77c-6e58-4a95-9863-4aa33c88855d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:26.065765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a77c-6e58-4a95-9863-4aa33c88855d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:26.153729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 862/1722] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:25 2022] POST /identity/v3/users => generated 292 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:26.155790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a77c-6e58-4a95-9863-4aa33c88855d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:26.198902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 861/1723] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:26.226733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a77c-6e58-4a95-9863-4aa33c88855d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:26.256751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498a828-657e-4776-ab11-ec7c05e24c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:26.259577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498a828-657e-4776-ab11-ec7c05e24c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:26.260349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498a828-657e-4776-ab11-ec7c05e24c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:26.264361 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498a828-657e-4776-ab11-ec7c05e24c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:26.266556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498a828-657e-4776-ab11-ec7c05e24c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:26.288133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498a828-657e-4776-ab11-ec7c05e24c3a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:26.327536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 863/1724] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:37:26.331547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24295298-15c5-47d4-8ecc-ecf99ce7e311 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:26.356492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24295298-15c5-47d4-8ecc-ecf99ce7e311 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:26.371031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448f9e8-cd8d-4685-96fd-9fc19881cb6b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:26.371609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448f9e8-cd8d-4685-96fd-9fc19881cb6b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:26.372093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448f9e8-cd8d-4685-96fd-9fc19881cb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:26.385124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf772f9-f697-45ab-9f09-78c7ed7fe2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:26.389164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf772f9-f697-45ab-9f09-78c7ed7fe2e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:26.390042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf772f9-f697-45ab-9f09-78c7ed7fe2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:26.391295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf772f9-f697-45ab-9f09-78c7ed7fe2e5 admin admin] PATH_INFO: `/v3/projects/b543c58fabef4d9bae9d1d338e44b89b/users/1191b788be1a4ecd90e88a3ed3b11e4d/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:26.441327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf772f9-f697-45ab-9f09-78c7ed7fe2e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b543c58fabef4d9bae9d1d338e44b89b, user_id=1191b788be1a4ecd90e88a3ed3b11e4d, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:26.449406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf772f9-f697-45ab-9f09-78c7ed7fe2e5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:26.456878 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f448f9e8-cd8d-4685-96fd-9fc19881cb6b None None] MFA Rules not processed for user `1e02f414ef2a4990bd1492a63d074319`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:26.458686 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f448f9e8-cd8d-4685-96fd-9fc19881cb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:26.475977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:26.484867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 862/1725] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:26 2022] PUT /identity/v3/projects/b543c58fabef4d9bae9d1d338e44b89b/users/1191b788be1a4ecd90e88a3ed3b11e4d/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:26.488884 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf772f9-f697-45ab-9f09-78c7ed7fe2e5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:26.600807 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:26.624837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 864/1726] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:26.673260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf772f9-f697-45ab-9f09-78c7ed7fe2e5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:26.691466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07410f94-6448-4b08-a9ce-64392f1b31cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:26.692122 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07410f94-6448-4b08-a9ce-64392f1b31cf admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:26.692408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07410f94-6448-4b08-a9ce-64392f1b31cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:26.692690 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07410f94-6448-4b08-a9ce-64392f1b31cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:26.693593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07410f94-6448-4b08-a9ce-64392f1b31cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:26.695268 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07410f94-6448-4b08-a9ce-64392f1b31cf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:26.749714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 863/1727] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:26.751302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448f9e8-cd8d-4685-96fd-9fc19881cb6b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:26.786663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448f9e8-cd8d-4685-96fd-9fc19881cb6b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:26.797115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f24cad-01aa-4432-b824-eb66b611e9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:26.798549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f24cad-01aa-4432-b824-eb66b611e9e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:26.799258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f24cad-01aa-4432-b824-eb66b611e9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:26.803663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f24cad-01aa-4432-b824-eb66b611e9e9 admin admin] PATH_INFO: `/v3/projects/b543c58fabef4d9bae9d1d338e44b89b/users/1191b788be1a4ecd90e88a3ed3b11e4d/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:26.814043 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f24cad-01aa-4432-b824-eb66b611e9e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b543c58fabef4d9bae9d1d338e44b89b, user_id=1191b788be1a4ecd90e88a3ed3b11e4d, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:26.834790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f24cad-01aa-4432-b824-eb66b611e9e9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:26.867594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 865/1728] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:26 2022] PUT /identity/v3/projects/b543c58fabef4d9bae9d1d338e44b89b/users/1191b788be1a4ecd90e88a3ed3b11e4d/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:26.879094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c689c3f-5cbf-46aa-8704-f84b595d43be None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:26.879601 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c689c3f-5cbf-46aa-8704-f84b595d43be None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:26.879874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c689c3f-5cbf-46aa-8704-f84b595d43be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:26.902219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d0c90-088b-4148-80f0-2573db7d05c6 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:26.907731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d0c90-088b-4148-80f0-2573db7d05c6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:26.908313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d0c90-088b-4148-80f0-2573db7d05c6 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:26.913881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 866/1729] 10.222.0.40 () {54 vars in 961 bytes} [Mon Aug 1 07:37:26 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:37:26.920012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7790acb-76a1-45c5-9a69-7e595a3bb856 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:26.920576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7790acb-76a1-45c5-9a69-7e595a3bb856 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:26.921082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7790acb-76a1-45c5-9a69-7e595a3bb856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:26.967402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0c689c3f-5cbf-46aa-8704-f84b595d43be None None] MFA Rules not processed for user `1191b788be1a4ecd90e88a3ed3b11e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:26.968757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0c689c3f-5cbf-46aa-8704-f84b595d43be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:26.984810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d7790acb-76a1-45c5-9a69-7e595a3bb856 None None] MFA Rules not processed for user `928c5d1284b74f1291c031323bb03a20`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:26.986503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d7790acb-76a1-45c5-9a69-7e595a3bb856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:27.115506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 864/1730] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:27.116810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c689c3f-5cbf-46aa-8704-f84b595d43be None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:27.119137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0c689c3f-5cbf-46aa-8704-f84b595d43be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:27.150310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 867/1731] 10.222.0.40 () {58 vars in 1058 bytes} [Mon Aug 1 07:37:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:27.152369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7790acb-76a1-45c5-9a69-7e595a3bb856 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:27.153901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d7790acb-76a1-45c5-9a69-7e595a3bb856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:27.245231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c689c3f-5cbf-46aa-8704-f84b595d43be None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:27.255362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849580d-c145-4bbb-be2a-2b3446e56c44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:27.256264 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849580d-c145-4bbb-be2a-2b3446e56c44 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:27.262178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849580d-c145-4bbb-be2a-2b3446e56c44 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:27.262464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849580d-c145-4bbb-be2a-2b3446e56c44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:27.293419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7790acb-76a1-45c5-9a69-7e595a3bb856 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:27.298736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:27.299587 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:27.299868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:27.300145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] PATH_INFO: `/v3/projects/b21cc7e2f6e642f0895d707be28d8918` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:27.301898 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21cc7e2f6e642f0895d707be28d8918)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:27.305430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:27.306682 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:27.404746 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21cc7e2f6e642f0895d707be28d8918', 'request_id': 'req-5664941e-8dd6-498e-81fb-3662b4f1656b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:27.410156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849580d-c145-4bbb-be2a-2b3446e56c44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:27.413201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849580d-c145-4bbb-be2a-2b3446e56c44 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:27.414408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21cc7e2f6e642f0895d707be28d8918', 'request_id': 'req-5664941e-8dd6-498e-81fb-3662b4f1656b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:27.430132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21cc7e2f6e642f0895d707be28d8918', 'request_id': 'req-5664941e-8dd6-498e-81fb-3662b4f1656b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:27.432558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 865/1732] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:27.433568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849580d-c145-4bbb-be2a-2b3446e56c44 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:27.435882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d849580d-c145-4bbb-be2a-2b3446e56c44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:27.453318 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] The token cache is being invalidate because project b21cc7e2f6e642f0895d707be28d8918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:27.453768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:27.454519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:27.460543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 868/1733] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:27 2022] DELETE /identity/v3/projects/b21cc7e2f6e642f0895d707be28d8918 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:27.461798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:27.466693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:27.467518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849580d-c145-4bbb-be2a-2b3446e56c44 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:27.502383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:27.577550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664941e-8dd6-498e-81fb-3662b4f1656b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:27.581450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dab6e5-3501-4f60-9ce1-646b330ee9f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:27.582198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dab6e5-3501-4f60-9ce1-646b330ee9f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:27.582583 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dab6e5-3501-4f60-9ce1-646b330ee9f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:27.582958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dab6e5-3501-4f60-9ce1-646b330ee9f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:27.585423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-16dab6e5-3501-4f60-9ce1-646b330ee9f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:27.659795 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f88715-2960-466b-8aa8-4971ad1e5139 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928c5d1284b74f1291c031323bb03a20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:27.660698 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f88715-2960-466b-8aa8-4971ad1e5139 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:27.663963 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f88715-2960-466b-8aa8-4971ad1e5139 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:27.664407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f88715-2960-466b-8aa8-4971ad1e5139 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:27.667036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a6f88715-2960-466b-8aa8-4971ad1e5139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:27.702843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f88715-2960-466b-8aa8-4971ad1e5139 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:27.705591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f88715-2960-466b-8aa8-4971ad1e5139 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:27.719254 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 866/1734] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:27.720608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f88715-2960-466b-8aa8-4971ad1e5139 service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:27.723392 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a6f88715-2960-466b-8aa8-4971ad1e5139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:27.725482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dab6e5-3501-4f60-9ce1-646b330ee9f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:27.729107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dab6e5-3501-4f60-9ce1-646b330ee9f3 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:27.741097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f88715-2960-466b-8aa8-4971ad1e5139 service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:27.751562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 869/1735] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:27 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:27.756288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dab6e5-3501-4f60-9ce1-646b330ee9f3 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:27.756933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff244eba-7992-42d1-bced-fe78d22789d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928c5d1284b74f1291c031323bb03a20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:27.759168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff244eba-7992-42d1-bced-fe78d22789d5 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:27.759472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff244eba-7992-42d1-bced-fe78d22789d5 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:27.759755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff244eba-7992-42d1-bced-fe78d22789d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:27.760506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-16dab6e5-3501-4f60-9ce1-646b330ee9f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:27.789212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff244eba-7992-42d1-bced-fe78d22789d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:27.797128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff244eba-7992-42d1-bced-fe78d22789d5 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:27.839218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 867/1736] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:28.079831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dab6e5-3501-4f60-9ce1-646b330ee9f3 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:28.101497 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:28.102582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:28.103761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:28.104403 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] PATH_INFO: `/v3/users/d7df7a7188704f87912b082d37e21ae5` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:28.108663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7df7a7188704f87912b082d37e21ae5)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:28.115628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:28.238667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7df7a7188704f87912b082d37e21ae5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:28.284002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:28.285481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7df7a7188704f87912b082d37e21ae5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:28.286384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7df7a7188704f87912b082d37e21ae5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:28.304919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7df7a7188704f87912b082d37e21ae5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:28.331260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7df7a7188704f87912b082d37e21ae5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:28.355701 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 870/1737] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:27 2022] DELETE /identity/v3/users/d7df7a7188704f87912b082d37e21ae5 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:28.387766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173acd8b-9548-45bc-b8aa-c30afe3c39c8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:28.389030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173acd8b-9548-45bc-b8aa-c30afe3c39c8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:28.389468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173acd8b-9548-45bc-b8aa-c30afe3c39c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:28.449074 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-173acd8b-9548-45bc-b8aa-c30afe3c39c8 None None] MFA Rules not processed for user `db6ac4cf9dd84e338f944b4905dcdb50`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:28.451710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-173acd8b-9548-45bc-b8aa-c30afe3c39c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:28.629731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 868/1738] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:28.909355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:28.912476 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:29.297234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7a412-ebcc-439f-b06a-c6bb5d2bfdad admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:29.309039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173acd8b-9548-45bc-b8aa-c30afe3c39c8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:29.310915 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-173acd8b-9548-45bc-b8aa-c30afe3c39c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:29.335094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856769af-480e-446c-b6e1-061e1f1c04d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:29.339506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856769af-480e-446c-b6e1-061e1f1c04d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:29.342700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856769af-480e-446c-b6e1-061e1f1c04d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:29.343014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856769af-480e-446c-b6e1-061e1f1c04d0 admin admin] PATH_INFO: `/v3/users/e7fcfa754e5e4123a1515d487835496f` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:29.345953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173acd8b-9548-45bc-b8aa-c30afe3c39c8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:29.347495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856769af-480e-446c-b6e1-061e1f1c04d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7fcfa754e5e4123a1515d487835496f)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:29.366029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856769af-480e-446c-b6e1-061e1f1c04d0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:29.383210 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b406c-7f4e-4218-96f8-4107430cb9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:29.386560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b406c-7f4e-4218-96f8-4107430cb9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:29.388030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b406c-7f4e-4218-96f8-4107430cb9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:29.388569 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b406c-7f4e-4218-96f8-4107430cb9ce admin admin] PATH_INFO: `/v3/projects/a87b0cbceaac4a11a55a02d63737670e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:29.394537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b406c-7f4e-4218-96f8-4107430cb9ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=a87b0cbceaac4a11a55a02d63737670e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:29.405202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b406c-7f4e-4218-96f8-4107430cb9ce admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:29.531273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:29.531594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-856769af-480e-446c-b6e1-061e1f1c04d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7fcfa754e5e4123a1515d487835496f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:29.579691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-008b406c-7f4e-4218-96f8-4107430cb9ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a87b0cbceaac4a11a55a02d63737670e', 'request_id': 'req-008b406c-7f4e-4218-96f8-4107430cb9ce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:29.579907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-856769af-480e-446c-b6e1-061e1f1c04d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7fcfa754e5e4123a1515d487835496f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:29.582946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-856769af-480e-446c-b6e1-061e1f1c04d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7fcfa754e5e4123a1515d487835496f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:29.612226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-008b406c-7f4e-4218-96f8-4107430cb9ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a87b0cbceaac4a11a55a02d63737670e', 'request_id': 'req-008b406c-7f4e-4218-96f8-4107430cb9ce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:29.633791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-856769af-480e-446c-b6e1-061e1f1c04d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7fcfa754e5e4123a1515d487835496f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:29.654722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-008b406c-7f4e-4218-96f8-4107430cb9ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a87b0cbceaac4a11a55a02d63737670e', 'request_id': 'req-008b406c-7f4e-4218-96f8-4107430cb9ce'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:29.668272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-856769af-480e-446c-b6e1-061e1f1c04d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7fcfa754e5e4123a1515d487835496f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:29.695288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 871/1739] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:28 2022] DELETE /identity/v3/users/e7fcfa754e5e4123a1515d487835496f => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:29.718293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-008b406c-7f4e-4218-96f8-4107430cb9ce admin admin] The token cache is being invalidate because project a87b0cbceaac4a11a55a02d63737670e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:29.718293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-008b406c-7f4e-4218-96f8-4107430cb9ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:29.718690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc91673-f628-4c83-8c1b-b2918e92f01a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:29.719202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc91673-f628-4c83-8c1b-b2918e92f01a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:29.719556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc91673-f628-4c83-8c1b-b2918e92f01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:29.735384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-008b406c-7f4e-4218-96f8-4107430cb9ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:29.737715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 869/1740] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:29 2022] DELETE /identity/v3/projects/a87b0cbceaac4a11a55a02d63737670e => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:37:29.744623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c16680-3ea3-4d0e-8613-92c81cb8276d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:29.744957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c16680-3ea3-4d0e-8613-92c81cb8276d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:29.745204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c16680-3ea3-4d0e-8613-92c81cb8276d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:29.818403 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bcc91673-f628-4c83-8c1b-b2918e92f01a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:29.819726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bcc91673-f628-4c83-8c1b-b2918e92f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:29.836340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-11c16680-3ea3-4d0e-8613-92c81cb8276d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:29.837586 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-11c16680-3ea3-4d0e-8613-92c81cb8276d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:30.039191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:30.060155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 872/1741] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:30.070083 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc91673-f628-4c83-8c1b-b2918e92f01a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:30.071719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bcc91673-f628-4c83-8c1b-b2918e92f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:30.106453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 870/1742] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:30.114654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d6e65-604b-456d-bcbf-e40bb08a84c5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:30.117246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d6e65-604b-456d-bcbf-e40bb08a84c5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:30.118132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d6e65-604b-456d-bcbf-e40bb08a84c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:30.169787 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1a6d6e65-604b-456d-bcbf-e40bb08a84c5 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:30.178999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1a6d6e65-604b-456d-bcbf-e40bb08a84c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:30.206984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 871/1743] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:30.211688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d6e65-604b-456d-bcbf-e40bb08a84c5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:30.230713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d6e65-604b-456d-bcbf-e40bb08a84c5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:30.242235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fb10f-494c-4371-bb4b-f109f93b85c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:30.242617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fb10f-494c-4371-bb4b-f109f93b85c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:30.245667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fb10f-494c-4371-bb4b-f109f93b85c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:30.247110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fb10f-494c-4371-bb4b-f109f93b85c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:30.247970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fb10f-494c-4371-bb4b-f109f93b85c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:30.249557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fb10f-494c-4371-bb4b-f109f93b85c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:30.254806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fb10f-494c-4371-bb4b-f109f93b85c9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:30.296372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 872/1744] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:30.297790 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fb10f-494c-4371-bb4b-f109f93b85c9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:30.319881 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fb10f-494c-4371-bb4b-f109f93b85c9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:30.337916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc91673-f628-4c83-8c1b-b2918e92f01a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:30.339528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001453d-dff5-42e9-ab17-8a629ad21934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:30.340281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001453d-dff5-42e9-ab17-8a629ad21934 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:30.342935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001453d-dff5-42e9-ab17-8a629ad21934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:30.343503 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001453d-dff5-42e9-ab17-8a629ad21934 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:30.344212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001453d-dff5-42e9-ab17-8a629ad21934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:30.348025 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001453d-dff5-42e9-ab17-8a629ad21934 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:30.353537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420052d2-fac8-4ee8-94db-1ed90dd3ed61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:30.354916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420052d2-fac8-4ee8-94db-1ed90dd3ed61 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:30.355260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420052d2-fac8-4ee8-94db-1ed90dd3ed61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:30.355553 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420052d2-fac8-4ee8-94db-1ed90dd3ed61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:30.392822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420052d2-fac8-4ee8-94db-1ed90dd3ed61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:30.399365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420052d2-fac8-4ee8-94db-1ed90dd3ed61 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:30.425961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 873/1746] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:30 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:37:30.426446 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 873/1746] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:30.430437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420052d2-fac8-4ee8-94db-1ed90dd3ed61 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:30.487165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420052d2-fac8-4ee8-94db-1ed90dd3ed61 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:30.641002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfd585-4b87-404b-ae34-45b5b8d745df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:30.641876 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfd585-4b87-404b-ae34-45b5b8d745df admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:30.642235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfd585-4b87-404b-ae34-45b5b8d745df admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:30.642684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfd585-4b87-404b-ae34-45b5b8d745df admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:30.643683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfd585-4b87-404b-ae34-45b5b8d745df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:30.647115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfd585-4b87-404b-ae34-45b5b8d745df admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:30.703709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 874/1747] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:30 2022] POST /identity/v3/users => generated 289 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:30.709547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001453d-dff5-42e9-ab17-8a629ad21934 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:30.746971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001453d-dff5-42e9-ab17-8a629ad21934 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:30.774976 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed5ead-06b5-40af-9120-2ff5bfda3517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:30.776346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed5ead-06b5-40af-9120-2ff5bfda3517 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:30.776740 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed5ead-06b5-40af-9120-2ff5bfda3517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:30.777067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed5ead-06b5-40af-9120-2ff5bfda3517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:30.781785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed5ead-06b5-40af-9120-2ff5bfda3517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:30.785076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed5ead-06b5-40af-9120-2ff5bfda3517 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:30.823867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 874/1748] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:30.826034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfd585-4b87-404b-ae34-45b5b8d745df admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:30.861663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfd585-4b87-404b-ae34-45b5b8d745df admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:30.895644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66c4e7-5845-4653-8e25-40f2c0bc95f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:30.897115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66c4e7-5845-4653-8e25-40f2c0bc95f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:30.897625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66c4e7-5845-4653-8e25-40f2c0bc95f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:30.900935 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66c4e7-5845-4653-8e25-40f2c0bc95f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:30.908810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66c4e7-5845-4653-8e25-40f2c0bc95f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:30.910484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66c4e7-5845-4653-8e25-40f2c0bc95f8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:30.931252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 875/1749] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:37:30.933550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed5ead-06b5-40af-9120-2ff5bfda3517 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:30.980585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed5ead-06b5-40af-9120-2ff5bfda3517 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:30.989837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577fc9b-e21a-484e-92c4-438a77fd1fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:30.991535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577fc9b-e21a-484e-92c4-438a77fd1fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:30.991931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577fc9b-e21a-484e-92c4-438a77fd1fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:30.992313 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577fc9b-e21a-484e-92c4-438a77fd1fa4 admin admin] PATH_INFO: `/v3/projects/479b2b033c1c45a3bb419342a1d16027/users/276c3c1a8b26433298993f70984a7d8d/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:31.033784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577fc9b-e21a-484e-92c4-438a77fd1fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=479b2b033c1c45a3bb419342a1d16027, user_id=276c3c1a8b26433298993f70984a7d8d, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:31.039246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577fc9b-e21a-484e-92c4-438a77fd1fa4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:31.076682 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 875/1750] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:30 2022] PUT /identity/v3/projects/479b2b033c1c45a3bb419342a1d16027/users/276c3c1a8b26433298993f70984a7d8d/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:31.077681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de051c2-63d5-4542-a0e1-9523e0251ed0 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:31.078160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de051c2-63d5-4542-a0e1-9523e0251ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:31.078334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de051c2-63d5-4542-a0e1-9523e0251ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:31.167428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3de051c2-63d5-4542-a0e1-9523e0251ed0 None None] MFA Rules not processed for user `276c3c1a8b26433298993f70984a7d8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:31.169292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3de051c2-63d5-4542-a0e1-9523e0251ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:31.275197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 876/1751] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:31.286924 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577fc9b-e21a-484e-92c4-438a77fd1fa4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:31.289181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1577fc9b-e21a-484e-92c4-438a77fd1fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:31.353551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:31.400068 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577fc9b-e21a-484e-92c4-438a77fd1fa4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:31.406202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb57e54-e25d-4d6d-b361-78da4eb408ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f57fa108c3d478a9976f4aea5f33bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:31.406858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb57e54-e25d-4d6d-b361-78da4eb408ef service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:31.407180 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb57e54-e25d-4d6d-b361-78da4eb408ef service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:31.407518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb57e54-e25d-4d6d-b361-78da4eb408ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:31.420772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb57e54-e25d-4d6d-b361-78da4eb408ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:31.422925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb57e54-e25d-4d6d-b361-78da4eb408ef service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:31.438142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 876/1752] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:31.726271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de051c2-63d5-4542-a0e1-9523e0251ed0 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:31.727785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3de051c2-63d5-4542-a0e1-9523e0251ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:31.749217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:31.846611 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de051c2-63d5-4542-a0e1-9523e0251ed0 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:31.856297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e993d-7796-46ef-9deb-45243422b44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:31.857247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e993d-7796-46ef-9deb-45243422b44d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:31.857603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e993d-7796-46ef-9deb-45243422b44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:31.858235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e993d-7796-46ef-9deb-45243422b44d admin admin] PATH_INFO: `/v3/projects/b32dac66ce054e938e3d060e8a0f10a2` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:31.860071 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e993d-7796-46ef-9deb-45243422b44d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b32dac66ce054e938e3d060e8a0f10a2)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:31.862548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e993d-7796-46ef-9deb-45243422b44d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:31.989127 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-072e993d-7796-46ef-9deb-45243422b44d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32dac66ce054e938e3d060e8a0f10a2', 'request_id': 'req-072e993d-7796-46ef-9deb-45243422b44d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:32.002183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-072e993d-7796-46ef-9deb-45243422b44d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32dac66ce054e938e3d060e8a0f10a2', 'request_id': 'req-072e993d-7796-46ef-9deb-45243422b44d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:32.025193 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-072e993d-7796-46ef-9deb-45243422b44d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32dac66ce054e938e3d060e8a0f10a2', 'request_id': 'req-072e993d-7796-46ef-9deb-45243422b44d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:32.073983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-072e993d-7796-46ef-9deb-45243422b44d admin admin] The token cache is being invalidate because project b32dac66ce054e938e3d060e8a0f10a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:32.074763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-072e993d-7796-46ef-9deb-45243422b44d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:32.076343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-072e993d-7796-46ef-9deb-45243422b44d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:32.079616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 877/1753] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:31 2022] DELETE /identity/v3/projects/b32dac66ce054e938e3d060e8a0f10a2 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:32.084211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb57e54-e25d-4d6d-b361-78da4eb408ef service glance] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:32.100820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb57e54-e25d-4d6d-b361-78da4eb408ef service glance] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:32.318669 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:32.320517 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:32.324635 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:32.325028 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] PATH_INFO: `/v3/users/1191b788be1a4ecd90e88a3ed3b11e4d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:32.330220 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1191b788be1a4ecd90e88a3ed3b11e4d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:32.336071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:32.460901 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1191b788be1a4ecd90e88a3ed3b11e4d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:32.476971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1191b788be1a4ecd90e88a3ed3b11e4d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:32.497768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1191b788be1a4ecd90e88a3ed3b11e4d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:32.498822 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1191b788be1a4ecd90e88a3ed3b11e4d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:32.499769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1191b788be1a4ecd90e88a3ed3b11e4d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:32.517161 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 877/1754] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:32 2022] DELETE /identity/v3/users/1191b788be1a4ecd90e88a3ed3b11e4d => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:32.906357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5711468-d163-42f2-a347-19d8dde71f98 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:32.907992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5711468-d163-42f2-a347-19d8dde71f98 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:32.909226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5711468-d163-42f2-a347-19d8dde71f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:32.957306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c5711468-d163-42f2-a347-19d8dde71f98 None None] MFA Rules not processed for user `1e02f414ef2a4990bd1492a63d074319`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:32.959139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c5711468-d163-42f2-a347-19d8dde71f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:32.968668 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:32.970095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:33.056719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 878/1755] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:33.097016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:33.120303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4a401-b5f0-49e5-82d1-4f13f8d3e583 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:33.145396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:33.147651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:33.148091 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:33.148557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4 admin admin] PATH_INFO: `/v3/projects/b543c58fabef4d9bae9d1d338e44b89b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:33.151601 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b543c58fabef4d9bae9d1d338e44b89b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:33.153830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:33.268121 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b543c58fabef4d9bae9d1d338e44b89b', 'request_id': 'req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:33.280128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b543c58fabef4d9bae9d1d338e44b89b', 'request_id': 'req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:33.311384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b543c58fabef4d9bae9d1d338e44b89b', 'request_id': 'req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:33.356211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4 admin admin] The token cache is being invalidate because project b543c58fabef4d9bae9d1d338e44b89b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:33.356505 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:33.359760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f8de42d6-5c31-44ac-8d4c-7924ac5926b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:33.372351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 878/1756] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:32 2022] DELETE /identity/v3/projects/b543c58fabef4d9bae9d1d338e44b89b => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:37:33.373950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc901-f362-44a5-954f-2e3e1a577fbe None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:33.374460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc901-f362-44a5-954f-2e3e1a577fbe None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:33.374553 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc901-f362-44a5-954f-2e3e1a577fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:33.385089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e87f0-e482-4af3-9f29-c7478ef65801 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:33.385579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e87f0-e482-4af3-9f29-c7478ef65801 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:33.386011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e87f0-e482-4af3-9f29-c7478ef65801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:33.445052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cf1e87f0-e482-4af3-9f29-c7478ef65801 None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:33.446330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cf1e87f0-e482-4af3-9f29-c7478ef65801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:33.466531 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-748cc901-f362-44a5-954f-2e3e1a577fbe None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:33.471806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-748cc901-f362-44a5-954f-2e3e1a577fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:33.548098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 879/1757] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:37:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:33.584162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:33.624333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 879/1758] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:33.632690 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292527a-4d20-4c3f-b52a-cfc4530734d4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:33.633026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292527a-4d20-4c3f-b52a-cfc4530734d4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:33.633259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292527a-4d20-4c3f-b52a-cfc4530734d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:33.656010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c292527a-4d20-4c3f-b52a-cfc4530734d4 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:33.657382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c292527a-4d20-4c3f-b52a-cfc4530734d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:33.707091 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 880/1759] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:33.707832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cc901-f362-44a5-954f-2e3e1a577fbe None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:33.707832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-748cc901-f362-44a5-954f-2e3e1a577fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:33.712555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292527a-4d20-4c3f-b52a-cfc4530734d4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:33.746387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292527a-4d20-4c3f-b52a-cfc4530734d4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:33.762754 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f95a4-a276-4a72-ac8b-9b680daf7a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:33.763740 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f95a4-a276-4a72-ac8b-9b680daf7a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:33.766692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f95a4-a276-4a72-ac8b-9b680daf7a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:33.768259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f95a4-a276-4a72-ac8b-9b680daf7a98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:33.770623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f95a4-a276-4a72-ac8b-9b680daf7a98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:33.772158 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f95a4-a276-4a72-ac8b-9b680daf7a98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:33.777198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f95a4-a276-4a72-ac8b-9b680daf7a98 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:33.820239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 881/1760] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:33.826015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f95a4-a276-4a72-ac8b-9b680daf7a98 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:33.862586 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f95a4-a276-4a72-ac8b-9b680daf7a98 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:33.877123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e46071-c782-4303-9059-436789a5cb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:33.877900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e46071-c782-4303-9059-436789a5cb05 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:33.878264 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e46071-c782-4303-9059-436789a5cb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:33.878620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e46071-c782-4303-9059-436789a5cb05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:33.879416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e46071-c782-4303-9059-436789a5cb05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:33.882494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e46071-c782-4303-9059-436789a5cb05 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:33.962194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cc901-f362-44a5-954f-2e3e1a577fbe None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:33.969431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb31a6-eec8-4e3d-8489-920cc895faac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:33.970825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb31a6-eec8-4e3d-8489-920cc895faac service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:33.971343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb31a6-eec8-4e3d-8489-920cc895faac service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:33.971969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb31a6-eec8-4e3d-8489-920cc895faac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:33.985884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb31a6-eec8-4e3d-8489-920cc895faac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:33.987609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb31a6-eec8-4e3d-8489-920cc895faac service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:33.993562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 882/1761] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:33 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:33.994763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e46071-c782-4303-9059-436789a5cb05 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:34.017872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:34.048010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 880/1762] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:37:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:37:34.270814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e46071-c782-4303-9059-436789a5cb05 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:34.304049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b561ea9-9b91-4ed9-ac02-d6bbf40527d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:34.306964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b561ea9-9b91-4ed9-ac02-d6bbf40527d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:34.307225 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b561ea9-9b91-4ed9-ac02-d6bbf40527d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:34.311006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b561ea9-9b91-4ed9-ac02-d6bbf40527d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:34.313557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b561ea9-9b91-4ed9-ac02-d6bbf40527d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:34.329548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b561ea9-9b91-4ed9-ac02-d6bbf40527d4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:34.425120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 883/1763] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:33 2022] POST /identity/v3/users => generated 287 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:34.429526 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb31a6-eec8-4e3d-8489-920cc895faac service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:34.463201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb31a6-eec8-4e3d-8489-920cc895faac service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:34.490697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34deeebf-a16b-423d-89cc-7a5785036021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:34.496579 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34deeebf-a16b-423d-89cc-7a5785036021 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:34.499397 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34deeebf-a16b-423d-89cc-7a5785036021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:34.501591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34deeebf-a16b-423d-89cc-7a5785036021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:34.505419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34deeebf-a16b-423d-89cc-7a5785036021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:34.516452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34deeebf-a16b-423d-89cc-7a5785036021 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:34.545691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 881/1764] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:34.547978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b561ea9-9b91-4ed9-ac02-d6bbf40527d4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:34.570199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b561ea9-9b91-4ed9-ac02-d6bbf40527d4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:34.584039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833036d1-89f4-4572-af9c-6a49dbfc5210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:34.584690 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833036d1-89f4-4572-af9c-6a49dbfc5210 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:34.584993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833036d1-89f4-4572-af9c-6a49dbfc5210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:34.588863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833036d1-89f4-4572-af9c-6a49dbfc5210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:34.590759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833036d1-89f4-4572-af9c-6a49dbfc5210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:34.592811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833036d1-89f4-4572-af9c-6a49dbfc5210 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:34.637921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 884/1765] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:34.642580 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34deeebf-a16b-423d-89cc-7a5785036021 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:34.656628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34deeebf-a16b-423d-89cc-7a5785036021 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:34.677142 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0e97c-8f33-46e1-b277-6ce92d2bbc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:34.679008 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0e97c-8f33-46e1-b277-6ce92d2bbc3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:34.679008 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0e97c-8f33-46e1-b277-6ce92d2bbc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:34.679008 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0e97c-8f33-46e1-b277-6ce92d2bbc3f admin admin] PATH_INFO: `/v3/projects/a31ee41299df4ab3a0c40ab6997eb3da/users/1513d735a9b040c1b2cddb9a470d3e95/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:34.716546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0e97c-8f33-46e1-b277-6ce92d2bbc3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a31ee41299df4ab3a0c40ab6997eb3da, user_id=1513d735a9b040c1b2cddb9a470d3e95, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:34.721645 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0e97c-8f33-46e1-b277-6ce92d2bbc3f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:34.757744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 882/1766] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:34 2022] PUT /identity/v3/projects/a31ee41299df4ab3a0c40ab6997eb3da/users/1513d735a9b040c1b2cddb9a470d3e95/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:34.760425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5bb2b-0539-4446-913c-33d3f3f29243 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:34.761305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5bb2b-0539-4446-913c-33d3f3f29243 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:34.761947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5bb2b-0539-4446-913c-33d3f3f29243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:34.815941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dca5bb2b-0539-4446-913c-33d3f3f29243 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:34.817446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dca5bb2b-0539-4446-913c-33d3f3f29243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:34.930082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 885/1767] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:35.006470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0e97c-8f33-46e1-b277-6ce92d2bbc3f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:35.012454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-67b0e97c-8f33-46e1-b277-6ce92d2bbc3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:35.329161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0e97c-8f33-46e1-b277-6ce92d2bbc3f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:35.363845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27f85b-f1bb-458d-91ff-5db1845b29a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:35.365138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27f85b-f1bb-458d-91ff-5db1845b29a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:35.367136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27f85b-f1bb-458d-91ff-5db1845b29a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:35.367600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27f85b-f1bb-458d-91ff-5db1845b29a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:35.445226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27f85b-f1bb-458d-91ff-5db1845b29a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:35.446703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27f85b-f1bb-458d-91ff-5db1845b29a8 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:35.501818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 883/1768] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:35.813500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6268a-d510-438a-9813-a150bdd09957 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:35.813922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6268a-d510-438a-9813-a150bdd09957 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:35.814406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6268a-d510-438a-9813-a150bdd09957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:35.905403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bfe6268a-d510-438a-9813-a150bdd09957 None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:35.917864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bfe6268a-d510-438a-9813-a150bdd09957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:36.164395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 886/1769] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:37:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:36.198891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27f85b-f1bb-458d-91ff-5db1845b29a8 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:36.289613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:36.475717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27f85b-f1bb-458d-91ff-5db1845b29a8 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:36.497868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9c159-419d-4d43-9715-9a1151403ad4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:36.501809 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9c159-419d-4d43-9715-9a1151403ad4 service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:36.503576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9c159-419d-4d43-9715-9a1151403ad4 service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:36.504249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9c159-419d-4d43-9715-9a1151403ad4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:36.584076 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9c159-419d-4d43-9715-9a1151403ad4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:36.586028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9c159-419d-4d43-9715-9a1151403ad4 service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:36.674264 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 884/1770] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:37:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:37:38.477591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a609301-942f-4feb-93c5-fba52d9410c2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:38.478199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a609301-942f-4feb-93c5-fba52d9410c2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:38.478199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a609301-942f-4feb-93c5-fba52d9410c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:38.512845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9a609301-942f-4feb-93c5-fba52d9410c2 None None] MFA Rules not processed for user `1513d735a9b040c1b2cddb9a470d3e95`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:38.514985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9a609301-942f-4feb-93c5-fba52d9410c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:38.537894 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:38.600759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 887/1771] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:38.605149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9c159-419d-4d43-9715-9a1151403ad4 service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:38.615817 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9c159-419d-4d43-9715-9a1151403ad4 service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:38.621143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92999808-147f-4176-bb8d-225d221634df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:38.621885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92999808-147f-4176-bb8d-225d221634df admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:38.622236 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92999808-147f-4176-bb8d-225d221634df admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:38.622587 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92999808-147f-4176-bb8d-225d221634df admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:38.623433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92999808-147f-4176-bb8d-225d221634df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:38.625574 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92999808-147f-4176-bb8d-225d221634df admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:38.658685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 885/1772] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:38 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:37:38.659960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a609301-942f-4feb-93c5-fba52d9410c2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:38.672528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a609301-942f-4feb-93c5-fba52d9410c2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:38.796321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcee46-cc3b-4130-9506-af1aea1743fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:38.797105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcee46-cc3b-4130-9506-af1aea1743fd admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:38.797378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcee46-cc3b-4130-9506-af1aea1743fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:38.797659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcee46-cc3b-4130-9506-af1aea1743fd admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:38.798367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcee46-cc3b-4130-9506-af1aea1743fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:38.800312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcee46-cc3b-4130-9506-af1aea1743fd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:38.825407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 888/1773] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:38 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:38.826721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92999808-147f-4176-bb8d-225d221634df admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:38.838797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92999808-147f-4176-bb8d-225d221634df admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:38.843550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ae0f5-5c52-4e4f-9cc7-9b89e614e107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:38.844162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ae0f5-5c52-4e4f-9cc7-9b89e614e107 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:38.844677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ae0f5-5c52-4e4f-9cc7-9b89e614e107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:38.845003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ae0f5-5c52-4e4f-9cc7-9b89e614e107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:38.845867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ae0f5-5c52-4e4f-9cc7-9b89e614e107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:38.847492 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ae0f5-5c52-4e4f-9cc7-9b89e614e107 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:38.859889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 886/1774] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:38.864050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcee46-cc3b-4130-9506-af1aea1743fd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:38.874317 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcee46-cc3b-4130-9506-af1aea1743fd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:38.878946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44866b-3de2-4e45-a12d-110d2c863670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:38.879633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44866b-3de2-4e45-a12d-110d2c863670 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:38.879921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44866b-3de2-4e45-a12d-110d2c863670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:38.880520 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44866b-3de2-4e45-a12d-110d2c863670 admin admin] PATH_INFO: `/v3/projects/6100c8af2ca44801b2b6b84ac3f18b18/users/8d027f219bb345dca37c0ae3d9c0d8fb/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:38.895239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44866b-3de2-4e45-a12d-110d2c863670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6100c8af2ca44801b2b6b84ac3f18b18, user_id=8d027f219bb345dca37c0ae3d9c0d8fb, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:38.898499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44866b-3de2-4e45-a12d-110d2c863670 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:38.911842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 889/1775] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:38 2022] PUT /identity/v3/projects/6100c8af2ca44801b2b6b84ac3f18b18/users/8d027f219bb345dca37c0ae3d9c0d8fb/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:38.916370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ae0f5-5c52-4e4f-9cc7-9b89e614e107 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:38.926631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ae0f5-5c52-4e4f-9cc7-9b89e614e107 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:38.984235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94d4de-7665-41fa-90b9-f2ee0eb38a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:38.984854 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94d4de-7665-41fa-90b9-f2ee0eb38a47 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:38.985135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94d4de-7665-41fa-90b9-f2ee0eb38a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:38.985409 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94d4de-7665-41fa-90b9-f2ee0eb38a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:38.986097 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94d4de-7665-41fa-90b9-f2ee0eb38a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:38.987720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94d4de-7665-41fa-90b9-f2ee0eb38a47 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:38.997695 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 887/1776] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:39.001963 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44866b-3de2-4e45-a12d-110d2c863670 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:39.015533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44866b-3de2-4e45-a12d-110d2c863670 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:39.020391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d2c96-5040-4abc-a650-4594a90a9eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:39.021610 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d2c96-5040-4abc-a650-4594a90a9eda admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:39.022136 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d2c96-5040-4abc-a650-4594a90a9eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:39.022424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d2c96-5040-4abc-a650-4594a90a9eda admin admin] PATH_INFO: `/v3/projects/6100c8af2ca44801b2b6b84ac3f18b18/users/8d027f219bb345dca37c0ae3d9c0d8fb/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:39.024431 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d2c96-5040-4abc-a650-4594a90a9eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=6100c8af2ca44801b2b6b84ac3f18b18, user_id=8d027f219bb345dca37c0ae3d9c0d8fb, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:39.027120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d2c96-5040-4abc-a650-4594a90a9eda admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:39.056706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 890/1777] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:39 2022] PUT /identity/v3/projects/6100c8af2ca44801b2b6b84ac3f18b18/users/8d027f219bb345dca37c0ae3d9c0d8fb/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:40.033945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c9507-5be6-4549-80a0-254429dde74f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:40.034491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c9507-5be6-4549-80a0-254429dde74f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:40.034851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c9507-5be6-4549-80a0-254429dde74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:40.066100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-853c9507-5be6-4549-80a0-254429dde74f None None] MFA Rules not processed for user `8d027f219bb345dca37c0ae3d9c0d8fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:40.067539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-853c9507-5be6-4549-80a0-254429dde74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:40.202513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 888/1778] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:40.229000 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d2c96-5040-4abc-a650-4594a90a9eda admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:40.231562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7a2d2c96-5040-4abc-a650-4594a90a9eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:40.432146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d2c96-5040-4abc-a650-4594a90a9eda admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:40.446248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4795a07-a7fc-448b-a933-1a0ebae666cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:40.446908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795a07-a7fc-448b-a933-1a0ebae666cd service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:40.447224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795a07-a7fc-448b-a933-1a0ebae666cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:40.447506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795a07-a7fc-448b-a933-1a0ebae666cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:40.530051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:40.632383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4795a07-a7fc-448b-a933-1a0ebae666cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:40.634913 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4795a07-a7fc-448b-a933-1a0ebae666cd service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:40.665823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 891/1779] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:41.002543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c9507-5be6-4549-80a0-254429dde74f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:41.007169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-853c9507-5be6-4549-80a0-254429dde74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:41.022964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:41.187832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c9507-5be6-4549-80a0-254429dde74f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:41.193926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:41.194582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:41.195210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:41.195808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] PATH_INFO: `/v3/users/4e176fb300ae4e7b9d821ec54cf05cde` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:41.197709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e176fb300ae4e7b9d821ec54cf05cde)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:41.200329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:41.274788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e176fb300ae4e7b9d821ec54cf05cde'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:41.297383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e176fb300ae4e7b9d821ec54cf05cde'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:41.298144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e176fb300ae4e7b9d821ec54cf05cde'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:41.307528 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e176fb300ae4e7b9d821ec54cf05cde'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:41.317717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e176fb300ae4e7b9d821ec54cf05cde'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:41.325677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 889/1780] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:40 2022] DELETE /identity/v3/users/4e176fb300ae4e7b9d821ec54cf05cde => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:41.330313 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c141f6-bcc4-4193-ad68-b66ebed3fc1d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:41.330835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c141f6-bcc4-4193-ad68-b66ebed3fc1d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:41.330942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c141f6-bcc4-4193-ad68-b66ebed3fc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:41.366844 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-41c141f6-bcc4-4193-ad68-b66ebed3fc1d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:41.368049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-41c141f6-bcc4-4193-ad68-b66ebed3fc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:41.455969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 892/1781] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:41.465697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:41.467255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:41.560296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec913589-d714-4cda-ba73-2edb8949c34e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:41.565342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a60528-3cdf-4efc-a4ee-e4798245eb23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:41.579740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a60528-3cdf-4efc-a4ee-e4798245eb23 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:41.580035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a60528-3cdf-4efc-a4ee-e4798245eb23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:41.580355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a60528-3cdf-4efc-a4ee-e4798245eb23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:41.607476 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a60528-3cdf-4efc-a4ee-e4798245eb23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:41.609007 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a60528-3cdf-4efc-a4ee-e4798245eb23 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:41.637948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 890/1782] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:42.053530 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c141f6-bcc4-4193-ad68-b66ebed3fc1d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:42.055030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-41c141f6-bcc4-4193-ad68-b66ebed3fc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:42.068727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c141f6-bcc4-4193-ad68-b66ebed3fc1d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:42.086554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:42.087265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:42.087556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:42.088023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] PATH_INFO: `/v3/projects/78ff3c4979364b75a0746c0e904a53fc` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:42.089516 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ff3c4979364b75a0746c0e904a53fc)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:42.091599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:42.207924 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ff3c4979364b75a0746c0e904a53fc', 'request_id': 'req-72d7f373-406e-40c0-ad0f-720de76a4c54'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:42.254044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ff3c4979364b75a0746c0e904a53fc', 'request_id': 'req-72d7f373-406e-40c0-ad0f-720de76a4c54'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:42.266508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ff3c4979364b75a0746c0e904a53fc', 'request_id': 'req-72d7f373-406e-40c0-ad0f-720de76a4c54'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:42.287174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] The token cache is being invalidate because project 78ff3c4979364b75a0746c0e904a53fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:42.287479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:42.288142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:42.290900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 893/1783] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:42 2022] DELETE /identity/v3/projects/78ff3c4979364b75a0746c0e904a53fc => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:42.295009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a60528-3cdf-4efc-a4ee-e4798245eb23 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:42.318527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a60528-3cdf-4efc-a4ee-e4798245eb23 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:42.336249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:42.337714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:42.360659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:42.441903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:42.443167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:42.443919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:42.444196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] PATH_INFO: `/v3/users/c7bd3cafc6af4a0980701a3327b78f56` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:42.445646 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7bd3cafc6af4a0980701a3327b78f56)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:42.447621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:42.450712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7f373-406e-40c0-ad0f-720de76a4c54 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:42.471819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db2206-0261-496a-8641-52a0253963d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:42.472823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db2206-0261-496a-8641-52a0253963d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:42.473207 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db2206-0261-496a-8641-52a0253963d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:42.473553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db2206-0261-496a-8641-52a0253963d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:42.480786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-99db2206-0261-496a-8641-52a0253963d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:42.486613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:42.538659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7bd3cafc6af4a0980701a3327b78f56'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:42.566804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7bd3cafc6af4a0980701a3327b78f56'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:42.569504 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7bd3cafc6af4a0980701a3327b78f56'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:42.583695 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7bd3cafc6af4a0980701a3327b78f56'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:42.592333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7bd3cafc6af4a0980701a3327b78f56'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:42.597448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db2206-0261-496a-8641-52a0253963d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:42.599391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db2206-0261-496a-8641-52a0253963d8 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:42.603108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 891/1784] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:42 2022] DELETE /identity/v3/users/c7bd3cafc6af4a0980701a3327b78f56 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:42.613417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 894/1785] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:37:43.072405 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:43.073980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:43.153564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cac0c-1f27-4d2a-8104-ca7973fbef4a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:43.158090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:43.158732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:43.159020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:43.159362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0 admin admin] PATH_INFO: `/v3/projects/df73b49fde7a48108a4cb6e6b36957aa` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:43.160557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df73b49fde7a48108a4cb6e6b36957aa)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:43.162937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:43.207815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:43.212460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df73b49fde7a48108a4cb6e6b36957aa', 'request_id': 'req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:43.220279 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df73b49fde7a48108a4cb6e6b36957aa', 'request_id': 'req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:43.228766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df73b49fde7a48108a4cb6e6b36957aa', 'request_id': 'req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:43.242996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0 admin admin] The token cache is being invalidate because project df73b49fde7a48108a4cb6e6b36957aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:43.243315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:43.243944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5e74f45a-3f57-42e8-9aa1-88a866fe28a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:43.245464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 892/1786] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:43 2022] DELETE /identity/v3/projects/df73b49fde7a48108a4cb6e6b36957aa => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:43.250326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715f44b-0caa-4028-a2d7-5814f1a76197 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:43.250619 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715f44b-0caa-4028-a2d7-5814f1a76197 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:43.250905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715f44b-0caa-4028-a2d7-5814f1a76197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:43.271721 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7715f44b-0caa-4028-a2d7-5814f1a76197 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:43.273115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7715f44b-0caa-4028-a2d7-5814f1a76197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:43.351231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 895/1787] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:43.357419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5c9a1-61a6-4726-8749-63b9b3cb236f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:43.358045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5c9a1-61a6-4726-8749-63b9b3cb236f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:43.358437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5c9a1-61a6-4726-8749-63b9b3cb236f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:43.384585 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8db5c9a1-61a6-4726-8749-63b9b3cb236f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:43.386126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8db5c9a1-61a6-4726-8749-63b9b3cb236f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:43.413680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 893/1788] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:43.418737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715f44b-0caa-4028-a2d7-5814f1a76197 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:43.434278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715f44b-0caa-4028-a2d7-5814f1a76197 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:43.440760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbcdbd8-a33b-4c46-89b4-7d009d47336a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:43.441495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbcdbd8-a33b-4c46-89b4-7d009d47336a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:43.441786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbcdbd8-a33b-4c46-89b4-7d009d47336a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:43.442093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbcdbd8-a33b-4c46-89b4-7d009d47336a admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:43.443523 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbcdbd8-a33b-4c46-89b4-7d009d47336a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:43.443842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbcdbd8-a33b-4c46-89b4-7d009d47336a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:43.445452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbcdbd8-a33b-4c46-89b4-7d009d47336a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:43.459483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 896/1789] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:43.463949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5c9a1-61a6-4726-8749-63b9b3cb236f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:43.473213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5c9a1-61a6-4726-8749-63b9b3cb236f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:43.480325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282b040-d334-4392-b3d1-08c88dcc0cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:43.481073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282b040-d334-4392-b3d1-08c88dcc0cea admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:43.481382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282b040-d334-4392-b3d1-08c88dcc0cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:43.482090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282b040-d334-4392-b3d1-08c88dcc0cea admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:43.482881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282b040-d334-4392-b3d1-08c88dcc0cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:43.485026 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282b040-d334-4392-b3d1-08c88dcc0cea admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:43.526318 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 894/1790] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:43 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:43.531121 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbcdbd8-a33b-4c46-89b4-7d009d47336a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:43.542720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbcdbd8-a33b-4c46-89b4-7d009d47336a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:43.600379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9973f0-01dd-4c5a-8e79-ea0c4d38b9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:43.601037 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9973f0-01dd-4c5a-8e79-ea0c4d38b9f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:43.601382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9973f0-01dd-4c5a-8e79-ea0c4d38b9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:43.601663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9973f0-01dd-4c5a-8e79-ea0c4d38b9f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:43.602383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9973f0-01dd-4c5a-8e79-ea0c4d38b9f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:43.604563 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9973f0-01dd-4c5a-8e79-ea0c4d38b9f0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:43.625623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 897/1791] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:43 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:43.629826 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282b040-d334-4392-b3d1-08c88dcc0cea admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:43.641068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282b040-d334-4392-b3d1-08c88dcc0cea admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:43.645718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d188b9-67ae-473e-ad5f-6a0313bdd3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:43.647328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d188b9-67ae-473e-ad5f-6a0313bdd3ef admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:43.647640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d188b9-67ae-473e-ad5f-6a0313bdd3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:43.647942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d188b9-67ae-473e-ad5f-6a0313bdd3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:43.648684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d188b9-67ae-473e-ad5f-6a0313bdd3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:43.650395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d188b9-67ae-473e-ad5f-6a0313bdd3ef admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:43.660911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 895/1792] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:43.664842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9973f0-01dd-4c5a-8e79-ea0c4d38b9f0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:43.674860 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9973f0-01dd-4c5a-8e79-ea0c4d38b9f0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:43.679925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd920b-c76c-469b-9f50-5d4f5a04d933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:43.680539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd920b-c76c-469b-9f50-5d4f5a04d933 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:43.680821 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd920b-c76c-469b-9f50-5d4f5a04d933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:43.681099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd920b-c76c-469b-9f50-5d4f5a04d933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:43.681785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd920b-c76c-469b-9f50-5d4f5a04d933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:43.683424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd920b-c76c-469b-9f50-5d4f5a04d933 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:43.695112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 898/1793] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:37:43.699389 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d188b9-67ae-473e-ad5f-6a0313bdd3ef admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:43.710768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d188b9-67ae-473e-ad5f-6a0313bdd3ef admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:43.715425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0ba25-50c3-461f-b4a6-11a230ed8497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:43.716086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0ba25-50c3-461f-b4a6-11a230ed8497 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:43.716377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0ba25-50c3-461f-b4a6-11a230ed8497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:43.716659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0ba25-50c3-461f-b4a6-11a230ed8497 admin admin] PATH_INFO: `/v3/projects/6145ff1ba4304f4b9517a8b53279d6a6/users/0c1d05730cd943cea9a2fd10b27f7280/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:43.732684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0ba25-50c3-461f-b4a6-11a230ed8497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6145ff1ba4304f4b9517a8b53279d6a6, user_id=0c1d05730cd943cea9a2fd10b27f7280, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:43.735418 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0ba25-50c3-461f-b4a6-11a230ed8497 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:43.756341 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 896/1794] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:43 2022] PUT /identity/v3/projects/6145ff1ba4304f4b9517a8b53279d6a6/users/0c1d05730cd943cea9a2fd10b27f7280/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:43.766774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81cf89-aedc-4a2f-a7ff-6987b4c7568d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:43.767185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81cf89-aedc-4a2f-a7ff-6987b4c7568d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:43.767472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81cf89-aedc-4a2f-a7ff-6987b4c7568d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:43.780722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:43.795750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2e81cf89-aedc-4a2f-a7ff-6987b4c7568d None None] MFA Rules not processed for user `0c1d05730cd943cea9a2fd10b27f7280`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:43.796969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2e81cf89-aedc-4a2f-a7ff-6987b4c7568d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:43.869403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 899/1795] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:43.873848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0ba25-50c3-461f-b4a6-11a230ed8497 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:43.947533 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0ba25-50c3-461f-b4a6-11a230ed8497 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:43.953922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ef6a9-cfe9-4a2f-9e4e-e5896a8f6c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:43.954558 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ef6a9-cfe9-4a2f-9e4e-e5896a8f6c26 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:43.954940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ef6a9-cfe9-4a2f-9e4e-e5896a8f6c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:43.955253 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ef6a9-cfe9-4a2f-9e4e-e5896a8f6c26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:43.955940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ef6a9-cfe9-4a2f-9e4e-e5896a8f6c26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:43.958883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ef6a9-cfe9-4a2f-9e4e-e5896a8f6c26 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:43.997173 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 897/1796] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:43 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:43.998691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81cf89-aedc-4a2f-a7ff-6987b4c7568d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:44.011116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81cf89-aedc-4a2f-a7ff-6987b4c7568d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:44.087347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f250714-2ae6-49a4-9ae1-72ed2b3f07fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:44.089296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f250714-2ae6-49a4-9ae1-72ed2b3f07fd admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:44.089296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f250714-2ae6-49a4-9ae1-72ed2b3f07fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:44.089296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f250714-2ae6-49a4-9ae1-72ed2b3f07fd admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:44.089296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f250714-2ae6-49a4-9ae1-72ed2b3f07fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:44.091587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f250714-2ae6-49a4-9ae1-72ed2b3f07fd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:44.135816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 900/1797] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:43 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:37:44.139648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ef6a9-cfe9-4a2f-9e4e-e5896a8f6c26 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:44.152000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ef6a9-cfe9-4a2f-9e4e-e5896a8f6c26 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:44.158571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec6794-325a-4744-af03-c4c3de0228e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:44.158571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec6794-325a-4744-af03-c4c3de0228e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:44.158571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec6794-325a-4744-af03-c4c3de0228e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:44.158571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec6794-325a-4744-af03-c4c3de0228e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:44.159215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec6794-325a-4744-af03-c4c3de0228e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:44.160771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec6794-325a-4744-af03-c4c3de0228e4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:44.175134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 898/1798] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:37:44.177457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f250714-2ae6-49a4-9ae1-72ed2b3f07fd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:44.189148 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f250714-2ae6-49a4-9ae1-72ed2b3f07fd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:44.200174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1521e78-5a0e-4b36-84b6-96378836d404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:44.200875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1521e78-5a0e-4b36-84b6-96378836d404 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:44.201210 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1521e78-5a0e-4b36-84b6-96378836d404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:44.201503 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1521e78-5a0e-4b36-84b6-96378836d404 admin admin] PATH_INFO: `/v3/projects/35a657d0317b4d5b843c5b275aaffcca/users/32e0c1804a26459184f8bca1c5ffdb8c/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:44.219771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1521e78-5a0e-4b36-84b6-96378836d404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a657d0317b4d5b843c5b275aaffcca, user_id=32e0c1804a26459184f8bca1c5ffdb8c, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:44.222636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1521e78-5a0e-4b36-84b6-96378836d404 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:44.243170 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 901/1799] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:44 2022] PUT /identity/v3/projects/35a657d0317b4d5b843c5b275aaffcca/users/32e0c1804a26459184f8bca1c5ffdb8c/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:44.245144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec6794-325a-4744-af03-c4c3de0228e4 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:44.255854 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec6794-325a-4744-af03-c4c3de0228e4 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:44.313610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:44.331909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054d5ae-9573-43c1-acd9-e8af1ef44aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:44.332535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054d5ae-9573-43c1-acd9-e8af1ef44aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:44.332827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054d5ae-9573-43c1-acd9-e8af1ef44aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:44.333725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054d5ae-9573-43c1-acd9-e8af1ef44aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:44.334657 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054d5ae-9573-43c1-acd9-e8af1ef44aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:44.336416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054d5ae-9573-43c1-acd9-e8af1ef44aa8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:44.346832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 899/1800] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:44.351335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1521e78-5a0e-4b36-84b6-96378836d404 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:44.361447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1521e78-5a0e-4b36-84b6-96378836d404 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:44.366018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0add279-0f25-4b90-a432-0adbc4040ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:44.366676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0add279-0f25-4b90-a432-0adbc4040ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:44.366957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0add279-0f25-4b90-a432-0adbc4040ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:44.367266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0add279-0f25-4b90-a432-0adbc4040ec7 admin admin] PATH_INFO: `/v3/projects/35a657d0317b4d5b843c5b275aaffcca/users/32e0c1804a26459184f8bca1c5ffdb8c/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:44.369180 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0add279-0f25-4b90-a432-0adbc4040ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a657d0317b4d5b843c5b275aaffcca, user_id=32e0c1804a26459184f8bca1c5ffdb8c, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:44.371828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0add279-0f25-4b90-a432-0adbc4040ec7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:44.382457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 902/1801] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:44 2022] PUT /identity/v3/projects/35a657d0317b4d5b843c5b275aaffcca/users/32e0c1804a26459184f8bca1c5ffdb8c/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:44.398161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89a321-3314-4299-ac23-20de50d3a866 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:44.398787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89a321-3314-4299-ac23-20de50d3a866 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:44.398787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89a321-3314-4299-ac23-20de50d3a866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:44.433397 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-bd89a321-3314-4299-ac23-20de50d3a866 None None] MFA Rules not processed for user `32e0c1804a26459184f8bca1c5ffdb8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:44.434858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bd89a321-3314-4299-ac23-20de50d3a866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:44.520117 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 900/1802] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:44.531997 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0add279-0f25-4b90-a432-0adbc4040ec7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:44.533379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a0add279-0f25-4b90-a432-0adbc4040ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:44.598596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:44.604655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0add279-0f25-4b90-a432-0adbc4040ec7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:44.610030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab8a4f-604d-465c-9cb4-5850cc79379e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:44.610787 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab8a4f-604d-465c-9cb4-5850cc79379e service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:44.611149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab8a4f-604d-465c-9cb4-5850cc79379e service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:44.611543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab8a4f-604d-465c-9cb4-5850cc79379e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:44.622560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab8a4f-604d-465c-9cb4-5850cc79379e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:44.624211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab8a4f-604d-465c-9cb4-5850cc79379e service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:44.634267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 903/1803] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:44.725783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89a321-3314-4299-ac23-20de50d3a866 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:44.735251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89a321-3314-4299-ac23-20de50d3a866 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:44.738836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f4076-e0c3-4e0c-9d50-569a24bc48ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:44.740304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f4076-e0c3-4e0c-9d50-569a24bc48ec service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:44.740585 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f4076-e0c3-4e0c-9d50-569a24bc48ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:44.740864 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f4076-e0c3-4e0c-9d50-569a24bc48ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:44.809275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f4076-e0c3-4e0c-9d50-569a24bc48ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:44.810654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f4076-e0c3-4e0c-9d50-569a24bc48ec service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:44.819605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 901/1804] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:48.608697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efff3a-237d-4a95-868e-a639e0acd27f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:48.609188 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efff3a-237d-4a95-868e-a639e0acd27f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:48.609268 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efff3a-237d-4a95-868e-a639e0acd27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:48.648414 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-63efff3a-237d-4a95-868e-a639e0acd27f None None] MFA Rules not processed for user `138ede2854894a3b995435980965911c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:48.650417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-63efff3a-237d-4a95-868e-a639e0acd27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:48.776424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 904/1805] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Aug 1 07:37:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:48.931393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f4076-e0c3-4e0c-9d50-569a24bc48ec service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:48.931914 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efff3a-237d-4a95-868e-a639e0acd27f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:48.933367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-63efff3a-237d-4a95-868e-a639e0acd27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:48.934518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2d9f4076-e0c3-4e0c-9d50-569a24bc48ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:48.971295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f4076-e0c3-4e0c-9d50-569a24bc48ec service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:48.981879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d8841-0d06-445f-b0ca-cd500f103b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:48.982530 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d8841-0d06-445f-b0ca-cd500f103b2b service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:48.982818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d8841-0d06-445f-b0ca-cd500f103b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:48.983111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d8841-0d06-445f-b0ca-cd500f103b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:48.983529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efff3a-237d-4a95-868e-a639e0acd27f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:49.000886 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883e3c9-e4ab-41d2-98f2-1a5f39a85cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:49.001528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883e3c9-e4ab-41d2-98f2-1a5f39a85cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:49.001812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883e3c9-e4ab-41d2-98f2-1a5f39a85cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:49.002092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883e3c9-e4ab-41d2-98f2-1a5f39a85cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:49.007693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d8841-0d06-445f-b0ca-cd500f103b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:49.009428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d8841-0d06-445f-b0ca-cd500f103b2b service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:49.033501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883e3c9-e4ab-41d2-98f2-1a5f39a85cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:49.034960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883e3c9-e4ab-41d2-98f2-1a5f39a85cf9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:49.035334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 902/1806] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:49.077041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 905/1807] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:50.693634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d8841-0d06-445f-b0ca-cd500f103b2b service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:50.695937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1b3d8841-0d06-445f-b0ca-cd500f103b2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:50.781713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:50.836394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d8841-0d06-445f-b0ca-cd500f103b2b service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:50.847218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:50.848287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:50.849303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:50.849575 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] PATH_INFO: `/v3/users/5ab869e1fc0847bb87fdd64859141166` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:50.851356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ab869e1fc0847bb87fdd64859141166)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:50.852922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:50.985713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ab869e1fc0847bb87fdd64859141166'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:51.025230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ab869e1fc0847bb87fdd64859141166'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:51.026557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ab869e1fc0847bb87fdd64859141166'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:51.068453 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ab869e1fc0847bb87fdd64859141166'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:51.084581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ab869e1fc0847bb87fdd64859141166'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:51.098174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 903/1808] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:50 2022] DELETE /identity/v3/users/5ab869e1fc0847bb87fdd64859141166 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:51.101123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f95c6f-8ff4-4537-88c6-255390b2dd6b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:51.101608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f95c6f-8ff4-4537-88c6-255390b2dd6b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:51.101813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f95c6f-8ff4-4537-88c6-255390b2dd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:51.123422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-32f95c6f-8ff4-4537-88c6-255390b2dd6b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:51.124633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-32f95c6f-8ff4-4537-88c6-255390b2dd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:51.189260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 906/1809] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:51.200055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:51.202623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:51.289854 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479ff65-44cb-4d0d-a79a-f80055780b2d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:51.295712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fae47d-cd94-472f-9fc4-69078ca35da1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:51.296658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fae47d-cd94-472f-9fc4-69078ca35da1 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:51.297455 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fae47d-cd94-472f-9fc4-69078ca35da1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:51.297917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fae47d-cd94-472f-9fc4-69078ca35da1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:51.311553 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fae47d-cd94-472f-9fc4-69078ca35da1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:51.313912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fae47d-cd94-472f-9fc4-69078ca35da1 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:51.331295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 904/1810] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:51.510138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971383f4-6238-40a2-9bc0-51a677a39d72 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:51.510433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971383f4-6238-40a2-9bc0-51a677a39d72 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:51.510907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971383f4-6238-40a2-9bc0-51a677a39d72 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:51.512539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 907/1811] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:37:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:37:51.733761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eef0c2-9f03-4e90-be2e-04eb9e855ef5 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:51.733761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eef0c2-9f03-4e90-be2e-04eb9e855ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:51.734294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eef0c2-9f03-4e90-be2e-04eb9e855ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:51.757733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-88eef0c2-9f03-4e90-be2e-04eb9e855ef5 None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:51.758474 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-88eef0c2-9f03-4e90-be2e-04eb9e855ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:51.837081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971383f4-6238-40a2-9bc0-51a677a39d72 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:51.839587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-971383f4-6238-40a2-9bc0-51a677a39d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:51.850714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 905/1812] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:37:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:51.864737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971383f4-6238-40a2-9bc0-51a677a39d72 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:51.870120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:51.871088 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:51.871828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:51.872063 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] PATH_INFO: `/v3/projects/c0e0dd79b53841b1be8b1a4cedad91b1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:51.873323 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0e0dd79b53841b1be8b1a4cedad91b1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:51.874339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eef0c2-9f03-4e90-be2e-04eb9e855ef5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:51.875895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-88eef0c2-9f03-4e90-be2e-04eb9e855ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:51.876012 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:51.889870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:51.894668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eef0c2-9f03-4e90-be2e-04eb9e855ef5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:51.911372 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7288e46-7353-4024-984a-81a1ddf32b5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:51.912371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7288e46-7353-4024-984a-81a1ddf32b5a service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:51.912759 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7288e46-7353-4024-984a-81a1ddf32b5a service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:51.913142 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7288e46-7353-4024-984a-81a1ddf32b5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:51.929181 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7288e46-7353-4024-984a-81a1ddf32b5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:51.931215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7288e46-7353-4024-984a-81a1ddf32b5a service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:51.945193 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 906/1813] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:51.959880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0e0dd79b53841b1be8b1a4cedad91b1', 'request_id': 'req-c7c67aa7-019c-4fc2-a578-b1fdc322c455'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:51.971114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0e0dd79b53841b1be8b1a4cedad91b1', 'request_id': 'req-c7c67aa7-019c-4fc2-a578-b1fdc322c455'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:51.979481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0e0dd79b53841b1be8b1a4cedad91b1', 'request_id': 'req-c7c67aa7-019c-4fc2-a578-b1fdc322c455'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:51.992745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] The token cache is being invalidate because project c0e0dd79b53841b1be8b1a4cedad91b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:51.992963 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:51.993663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:51.995331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 908/1814] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:51 2022] DELETE /identity/v3/projects/c0e0dd79b53841b1be8b1a4cedad91b1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:52.000035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7288e46-7353-4024-984a-81a1ddf32b5a service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:52.014735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7288e46-7353-4024-984a-81a1ddf32b5a service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:52.101216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c9baf-6308-428a-9cc5-07773cedb88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:52.102202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c9baf-6308-428a-9cc5-07773cedb88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:52.102753 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c9baf-6308-428a-9cc5-07773cedb88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:52.103288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c9baf-6308-428a-9cc5-07773cedb88f admin admin] PATH_INFO: `/v3/users/f546368db1d4491eb95dd5b08b13343c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:52.104717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c9baf-6308-428a-9cc5-07773cedb88f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f546368db1d4491eb95dd5b08b13343c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:52.107318 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c9baf-6308-428a-9cc5-07773cedb88f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:52.172730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d2c9baf-6308-428a-9cc5-07773cedb88f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f546368db1d4491eb95dd5b08b13343c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:52.185869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d2c9baf-6308-428a-9cc5-07773cedb88f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f546368db1d4491eb95dd5b08b13343c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:52.186591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d2c9baf-6308-428a-9cc5-07773cedb88f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f546368db1d4491eb95dd5b08b13343c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:52.195253 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d2c9baf-6308-428a-9cc5-07773cedb88f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f546368db1d4491eb95dd5b08b13343c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:52.203250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d2c9baf-6308-428a-9cc5-07773cedb88f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f546368db1d4491eb95dd5b08b13343c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:52.215384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 907/1815] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:51 2022] DELETE /identity/v3/users/f546368db1d4491eb95dd5b08b13343c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:52.577243 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:52.578751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:52.592715 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c67aa7-019c-4fc2-a578-b1fdc322c455 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:52.602798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:52.701899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058e347-39e7-4a62-ba06-61a4f1b238e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:52.702465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058e347-39e7-4a62-ba06-61a4f1b238e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:52.703329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058e347-39e7-4a62-ba06-61a4f1b238e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:52.703650 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058e347-39e7-4a62-ba06-61a4f1b238e2 admin admin] PATH_INFO: `/v3/projects/7fefa59fb83944babc1c24b48d3bcd93` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:52.705590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058e347-39e7-4a62-ba06-61a4f1b238e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fefa59fb83944babc1c24b48d3bcd93)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:52.708421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058e347-39e7-4a62-ba06-61a4f1b238e2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:52.768166 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0058e347-39e7-4a62-ba06-61a4f1b238e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fefa59fb83944babc1c24b48d3bcd93', 'request_id': 'req-0058e347-39e7-4a62-ba06-61a4f1b238e2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:52.777615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0058e347-39e7-4a62-ba06-61a4f1b238e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fefa59fb83944babc1c24b48d3bcd93', 'request_id': 'req-0058e347-39e7-4a62-ba06-61a4f1b238e2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:52.788951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0058e347-39e7-4a62-ba06-61a4f1b238e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fefa59fb83944babc1c24b48d3bcd93', 'request_id': 'req-0058e347-39e7-4a62-ba06-61a4f1b238e2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:52.801874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0058e347-39e7-4a62-ba06-61a4f1b238e2 admin admin] The token cache is being invalidate because project 7fefa59fb83944babc1c24b48d3bcd93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:52.802180 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0058e347-39e7-4a62-ba06-61a4f1b238e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:52.802829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0058e347-39e7-4a62-ba06-61a4f1b238e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:52.804891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 909/1816] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:52 2022] DELETE /identity/v3/projects/7fefa59fb83944babc1c24b48d3bcd93 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:52.809440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3e8ee-5b7a-4c49-a377-bb249bf22ef1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:52.809930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3e8ee-5b7a-4c49-a377-bb249bf22ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:52.810021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3e8ee-5b7a-4c49-a377-bb249bf22ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:52.836128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b1c3e8ee-5b7a-4c49-a377-bb249bf22ef1 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:52.837771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b1c3e8ee-5b7a-4c49-a377-bb249bf22ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:52.945944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 908/1817] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:52.955536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6736190-1906-48c2-9219-70b88df21e58 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:52.956063 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6736190-1906-48c2-9219-70b88df21e58 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:52.956535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6736190-1906-48c2-9219-70b88df21e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:52.989719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d6736190-1906-48c2-9219-70b88df21e58 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:52.992006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d6736190-1906-48c2-9219-70b88df21e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:53.002210 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 910/1818] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:53.007322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3e8ee-5b7a-4c49-a377-bb249bf22ef1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:53.019100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3e8ee-5b7a-4c49-a377-bb249bf22ef1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:53.026640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812717e-87c2-4010-82d9-f3035b413bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:53.027993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812717e-87c2-4010-82d9-f3035b413bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:53.028468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812717e-87c2-4010-82d9-f3035b413bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:53.028851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812717e-87c2-4010-82d9-f3035b413bc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:53.029833 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812717e-87c2-4010-82d9-f3035b413bc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:37:53.030436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812717e-87c2-4010-82d9-f3035b413bc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:53.032551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812717e-87c2-4010-82d9-f3035b413bc4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:53.055585 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:53.070765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 909/1819] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:37:53.072275 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6736190-1906-48c2-9219-70b88df21e58 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:53.090949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6736190-1906-48c2-9219-70b88df21e58 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:53.096332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23535-9550-40f6-9b82-a1cdc2c91994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:53.097224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23535-9550-40f6-9b82-a1cdc2c91994 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:53.097677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23535-9550-40f6-9b82-a1cdc2c91994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:53.098094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23535-9550-40f6-9b82-a1cdc2c91994 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:53.100263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b23535-9550-40f6-9b82-a1cdc2c91994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:53.102809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b23535-9550-40f6-9b82-a1cdc2c91994 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:53.156549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 911/1820] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:53 2022] POST /identity/v3/projects => generated 401 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:53.158017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812717e-87c2-4010-82d9-f3035b413bc4 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:53.187783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812717e-87c2-4010-82d9-f3035b413bc4 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:53.277795 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff84003-0013-4ea7-9155-045c26dba44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:53.278553 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff84003-0013-4ea7-9155-045c26dba44d admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:53.278938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff84003-0013-4ea7-9155-045c26dba44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:53.279347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff84003-0013-4ea7-9155-045c26dba44d admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:53.280132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff84003-0013-4ea7-9155-045c26dba44d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:53.282239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff84003-0013-4ea7-9155-045c26dba44d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:53.316307 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 910/1821] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:53 2022] POST /identity/v3/users => generated 301 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:37:53.321942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23535-9550-40f6-9b82-a1cdc2c91994 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:53.333766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23535-9550-40f6-9b82-a1cdc2c91994 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:53.341296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bb9b7-9347-4826-a0d1-d5590dfc1b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:53.341866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bb9b7-9347-4826-a0d1-d5590dfc1b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:53.342758 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bb9b7-9347-4826-a0d1-d5590dfc1b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:53.342758 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bb9b7-9347-4826-a0d1-d5590dfc1b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:53.344191 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bb9b7-9347-4826-a0d1-d5590dfc1b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:53.345958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bb9b7-9347-4826-a0d1-d5590dfc1b13 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:53.359923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 912/1822] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:53.364357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff84003-0013-4ea7-9155-045c26dba44d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:53.379201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff84003-0013-4ea7-9155-045c26dba44d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:53.388450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc62f0-7ef7-4fde-88da-bd626b9f2957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:53.389163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc62f0-7ef7-4fde-88da-bd626b9f2957 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:53.389523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc62f0-7ef7-4fde-88da-bd626b9f2957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:53.389873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc62f0-7ef7-4fde-88da-bd626b9f2957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:53.390652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc62f0-7ef7-4fde-88da-bd626b9f2957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:53.392544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc62f0-7ef7-4fde-88da-bd626b9f2957 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:53.405838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 911/1823] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:53.409634 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bb9b7-9347-4826-a0d1-d5590dfc1b13 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:53.420976 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bb9b7-9347-4826-a0d1-d5590dfc1b13 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:53.434601 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b83db6-5b35-4cfa-9994-94134fcdd6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:53.435938 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b83db6-5b35-4cfa-9994-94134fcdd6b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:53.436579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b83db6-5b35-4cfa-9994-94134fcdd6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:53.437177 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b83db6-5b35-4cfa-9994-94134fcdd6b7 admin admin] PATH_INFO: `/v3/projects/7501231f33de47bc8c851baa8bde1174/users/452c6f5043764780bf2533d69d79596d/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:53.464918 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b83db6-5b35-4cfa-9994-94134fcdd6b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7501231f33de47bc8c851baa8bde1174, user_id=452c6f5043764780bf2533d69d79596d, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:53.468183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b83db6-5b35-4cfa-9994-94134fcdd6b7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:53.494808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 913/1824] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:53 2022] PUT /identity/v3/projects/7501231f33de47bc8c851baa8bde1174/users/452c6f5043764780bf2533d69d79596d/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:37:53.510261 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c9722-4167-4095-8f19-d6451578faeb None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:53.510636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c9722-4167-4095-8f19-d6451578faeb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:53.511039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c9722-4167-4095-8f19-d6451578faeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:53.553178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-087c9722-4167-4095-8f19-d6451578faeb None None] MFA Rules not processed for user `452c6f5043764780bf2533d69d79596d`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:53.554499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-087c9722-4167-4095-8f19-d6451578faeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:53.739623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7717c-0868-4920-b32d-5b95bfdaf9ed None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:53.741023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7717c-0868-4920-b32d-5b95bfdaf9ed None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:53.741467 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7717c-0868-4920-b32d-5b95bfdaf9ed None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:53.743192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 914/1825] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:37:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 07:37:53.749426 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 912/1826] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:53.754057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7717c-0868-4920-b32d-5b95bfdaf9ed None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:53.826470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec492bf-76a2-4a4a-8070-78a503850b0b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:53.828397 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec492bf-76a2-4a4a-8070-78a503850b0b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:53.828783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec492bf-76a2-4a4a-8070-78a503850b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:53.842369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:53.867183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dec492bf-76a2-4a4a-8070-78a503850b0b None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:53.868934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dec492bf-76a2-4a4a-8070-78a503850b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:53.891075 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7717c-0868-4920-b32d-5b95bfdaf9ed None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:53.897417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15a7a4-6103-4fbf-9591-343e17a93dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:53.898098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a7a4-6103-4fbf-9591-343e17a93dfe admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:53.898393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a7a4-6103-4fbf-9591-343e17a93dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:53.898677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a7a4-6103-4fbf-9591-343e17a93dfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:53.899390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15a7a4-6103-4fbf-9591-343e17a93dfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:53.904325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15a7a4-6103-4fbf-9591-343e17a93dfe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:53.960160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 915/1827] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:37:53 2022] POST /identity/v3/projects => generated 401 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:37:53.968252 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15a7a4-6103-4fbf-9591-343e17a93dfe admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:54.003676 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:54.023820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 913/1828] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:37:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:37:54.077495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15a7a4-6103-4fbf-9591-343e17a93dfe admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:54.084830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70127f08-19e7-4a84-9553-c8e3e7ef76aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:54.085636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70127f08-19e7-4a84-9553-c8e3e7ef76aa admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:54.086632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70127f08-19e7-4a84-9553-c8e3e7ef76aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:54.086632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70127f08-19e7-4a84-9553-c8e3e7ef76aa admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:54.087348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70127f08-19e7-4a84-9553-c8e3e7ef76aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:54.090266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70127f08-19e7-4a84-9553-c8e3e7ef76aa admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:54.122868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 916/1829] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:37:53 2022] POST /identity/v3/users => generated 307 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:37:54.127223 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec492bf-76a2-4a4a-8070-78a503850b0b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:54.151606 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec492bf-76a2-4a4a-8070-78a503850b0b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:54.160070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ef108-2fc0-4e8d-8cf8-605b95a243a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:54.162810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ef108-2fc0-4e8d-8cf8-605b95a243a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:54.163470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ef108-2fc0-4e8d-8cf8-605b95a243a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:54.164368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ef108-2fc0-4e8d-8cf8-605b95a243a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:54.165471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ef108-2fc0-4e8d-8cf8-605b95a243a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:54.167489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ef108-2fc0-4e8d-8cf8-605b95a243a5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:54.180225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 914/1830] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:54.184083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70127f08-19e7-4a84-9553-c8e3e7ef76aa admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:54.197081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70127f08-19e7-4a84-9553-c8e3e7ef76aa admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:54.213227 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16345ba-1945-4d9e-b634-9a7f4cc136b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:54.213940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16345ba-1945-4d9e-b634-9a7f4cc136b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:54.214201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16345ba-1945-4d9e-b634-9a7f4cc136b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:54.214487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16345ba-1945-4d9e-b634-9a7f4cc136b3 admin admin] PATH_INFO: `/v3/projects/e81a7548322343feb188cd72fa6cace9/users/fb26172ff1524b9897bb13afde9166a0/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:54.236194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16345ba-1945-4d9e-b634-9a7f4cc136b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81a7548322343feb188cd72fa6cace9, user_id=fb26172ff1524b9897bb13afde9166a0, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:54.239842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16345ba-1945-4d9e-b634-9a7f4cc136b3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:54.262611 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 917/1831] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:54 2022] PUT /identity/v3/projects/e81a7548322343feb188cd72fa6cace9/users/fb26172ff1524b9897bb13afde9166a0/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:54.266574 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ef108-2fc0-4e8d-8cf8-605b95a243a5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:54.282274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ef108-2fc0-4e8d-8cf8-605b95a243a5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:54.364515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7722b-c784-4fa2-9131-23dd6e746193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:54.365404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7722b-c784-4fa2-9131-23dd6e746193 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:54.365694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7722b-c784-4fa2-9131-23dd6e746193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:54.365976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7722b-c784-4fa2-9131-23dd6e746193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:54.366880 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7722b-c784-4fa2-9131-23dd6e746193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:54.368705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7722b-c784-4fa2-9131-23dd6e746193 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:54.382553 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 915/1832] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:37:54.386405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16345ba-1945-4d9e-b634-9a7f4cc136b3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:54.404470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16345ba-1945-4d9e-b634-9a7f4cc136b3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:54.409621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641cd6e-66e3-4a26-aee3-7c2ddfbd9ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:54.410303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641cd6e-66e3-4a26-aee3-7c2ddfbd9ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:54.410591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641cd6e-66e3-4a26-aee3-7c2ddfbd9ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:54.411031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641cd6e-66e3-4a26-aee3-7c2ddfbd9ee6 admin admin] PATH_INFO: `/v3/projects/e81a7548322343feb188cd72fa6cace9/users/fb26172ff1524b9897bb13afde9166a0/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:54.413280 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641cd6e-66e3-4a26-aee3-7c2ddfbd9ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81a7548322343feb188cd72fa6cace9, user_id=fb26172ff1524b9897bb13afde9166a0, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:54.415979 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641cd6e-66e3-4a26-aee3-7c2ddfbd9ee6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:54.435860 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 918/1833] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:37:54 2022] PUT /identity/v3/projects/e81a7548322343feb188cd72fa6cace9/users/fb26172ff1524b9897bb13afde9166a0/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:37:54.438261 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d6bbd-f98a-48d2-a009-fdceb5074a85 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:54.438678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d6bbd-f98a-48d2-a009-fdceb5074a85 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:54.439087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d6bbd-f98a-48d2-a009-fdceb5074a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:54.471365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b24d6bbd-f98a-48d2-a009-fdceb5074a85 None None] MFA Rules not processed for user `fb26172ff1524b9897bb13afde9166a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:54.472694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b24d6bbd-f98a-48d2-a009-fdceb5074a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:54.554656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641cd6e-66e3-4a26-aee3-7c2ddfbd9ee6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:54.556711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f641cd6e-66e3-4a26-aee3-7c2ddfbd9ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:54.557296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 916/1834] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:54 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:54.568473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d6bbd-f98a-48d2-a009-fdceb5074a85 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:54.569888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b24d6bbd-f98a-48d2-a009-fdceb5074a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:54.667593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d6bbd-f98a-48d2-a009-fdceb5074a85 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:54.672165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641cd6e-66e3-4a26-aee3-7c2ddfbd9ee6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:54.678323 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4258d5-c2e0-4461-81a7-e50af2f0b616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:54.679104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4258d5-c2e0-4461-81a7-e50af2f0b616 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:54.679470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4258d5-c2e0-4461-81a7-e50af2f0b616 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:54.679847 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4258d5-c2e0-4461-81a7-e50af2f0b616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:54.686494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbe766-0121-48b6-a0bc-965709c181d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:54.688752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbe766-0121-48b6-a0bc-965709c181d8 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:54.689821 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbe766-0121-48b6-a0bc-965709c181d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:54.691162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbe766-0121-48b6-a0bc-965709c181d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:54.704505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4258d5-c2e0-4461-81a7-e50af2f0b616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:54.704719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbe766-0121-48b6-a0bc-965709c181d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:54.706445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4258d5-c2e0-4461-81a7-e50af2f0b616 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:54.706707 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbe766-0121-48b6-a0bc-965709c181d8 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:54.716399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:54.732287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 917/1836] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:54 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:37:54.734210 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 919/1836] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:37:54.944447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4258d5-c2e0-4461-81a7-e50af2f0b616 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:54.955926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4d4258d5-c2e0-4461-81a7-e50af2f0b616 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:54.986317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4258d5-c2e0-4461-81a7-e50af2f0b616 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:54.999495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753deb3c-b715-497f-bfc9-fc6a13853df7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:55.000211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753deb3c-b715-497f-bfc9-fc6a13853df7 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:55.001379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753deb3c-b715-497f-bfc9-fc6a13853df7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:55.001739 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753deb3c-b715-497f-bfc9-fc6a13853df7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:55.019569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:55.079768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753deb3c-b715-497f-bfc9-fc6a13853df7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:55.081894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753deb3c-b715-497f-bfc9-fc6a13853df7 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:55.097962 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 918/1837] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:56.492430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbe766-0121-48b6-a0bc-965709c181d8 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:56.493840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3cdbe766-0121-48b6-a0bc-965709c181d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:56.560122 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbe766-0121-48b6-a0bc-965709c181d8 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:56.565104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:56.565760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:56.566049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:56.566326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] PATH_INFO: `/v3/users/0c1d05730cd943cea9a2fd10b27f7280` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:56.567553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c1d05730cd943cea9a2fd10b27f7280)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:56.569471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:56.628101 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c1d05730cd943cea9a2fd10b27f7280'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:56.640556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c1d05730cd943cea9a2fd10b27f7280'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:56.656642 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c1d05730cd943cea9a2fd10b27f7280'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:56.657301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c1d05730cd943cea9a2fd10b27f7280'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:56.658032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c1d05730cd943cea9a2fd10b27f7280'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:56.683311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 920/1838] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:56 2022] DELETE /identity/v3/users/0c1d05730cd943cea9a2fd10b27f7280 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:37:56.683415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322158a5-9a4c-4d8b-8e3c-2b22fe451c9b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:56.683415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322158a5-9a4c-4d8b-8e3c-2b22fe451c9b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:56.683415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322158a5-9a4c-4d8b-8e3c-2b22fe451c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:56.705994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-322158a5-9a4c-4d8b-8e3c-2b22fe451c9b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:56.708568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-322158a5-9a4c-4d8b-8e3c-2b22fe451c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:56.880416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 919/1839] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:57.066034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:57.066968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:57.228757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0962d8e-2c61-4cb9-8882-d7a85a5e9039 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:57.238508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49a246-9162-4d94-81f4-d13123b38621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:57.240860 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a246-9162-4d94-81f4-d13123b38621 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:57.241208 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a246-9162-4d94-81f4-d13123b38621 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:57.242089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a246-9162-4d94-81f4-d13123b38621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:57.268451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49a246-9162-4d94-81f4-d13123b38621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:57.270676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49a246-9162-4d94-81f4-d13123b38621 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:57.292296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 921/1840] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:59.025911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322158a5-9a4c-4d8b-8e3c-2b22fe451c9b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:59.028004 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-322158a5-9a4c-4d8b-8e3c-2b22fe451c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:59.054321 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322158a5-9a4c-4d8b-8e3c-2b22fe451c9b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:59.074770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c022d-33ab-4f60-8c66-33d28ec1b4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:59.075780 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c022d-33ab-4f60-8c66-33d28ec1b4e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:59.076276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c022d-33ab-4f60-8c66-33d28ec1b4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:59.076558 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c022d-33ab-4f60-8c66-33d28ec1b4e5 admin admin] PATH_INFO: `/v3/projects/6145ff1ba4304f4b9517a8b53279d6a6` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:59.077979 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c022d-33ab-4f60-8c66-33d28ec1b4e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6145ff1ba4304f4b9517a8b53279d6a6)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:59.080420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c022d-33ab-4f60-8c66-33d28ec1b4e5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:59.206574 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-584c022d-33ab-4f60-8c66-33d28ec1b4e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6145ff1ba4304f4b9517a8b53279d6a6', 'request_id': 'req-584c022d-33ab-4f60-8c66-33d28ec1b4e5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:59.213965 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-584c022d-33ab-4f60-8c66-33d28ec1b4e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6145ff1ba4304f4b9517a8b53279d6a6', 'request_id': 'req-584c022d-33ab-4f60-8c66-33d28ec1b4e5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:59.227871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-584c022d-33ab-4f60-8c66-33d28ec1b4e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6145ff1ba4304f4b9517a8b53279d6a6', 'request_id': 'req-584c022d-33ab-4f60-8c66-33d28ec1b4e5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:59.243303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-584c022d-33ab-4f60-8c66-33d28ec1b4e5 admin admin] The token cache is being invalidate because project 6145ff1ba4304f4b9517a8b53279d6a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:37:59.243602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-584c022d-33ab-4f60-8c66-33d28ec1b4e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:59.244396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-584c022d-33ab-4f60-8c66-33d28ec1b4e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:59.246244 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 920/1841] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:59 2022] DELETE /identity/v3/projects/6145ff1ba4304f4b9517a8b53279d6a6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:37:59.251013 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49a246-9162-4d94-81f4-d13123b38621 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:59.265680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49a246-9162-4d94-81f4-d13123b38621 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:59.270511 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8060c-12a5-40fc-829d-a3985d68ada1 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:59.271031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8060c-12a5-40fc-829d-a3985d68ada1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:59.271573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8060c-12a5-40fc-829d-a3985d68ada1 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:59.273705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 921/1842] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:37:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:37:59.292207 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:59.293140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed2b2b-84fc-4b6a-ab6f-aee279a6809c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:59.295797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed2b2b-84fc-4b6a-ab6f-aee279a6809c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:59.296225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed2b2b-84fc-4b6a-ab6f-aee279a6809c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:59.321442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c6ed2b2b-84fc-4b6a-ab6f-aee279a6809c None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:37:59.322827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c6ed2b2b-84fc-4b6a-ab6f-aee279a6809c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:59.349616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfde528-89b5-4ae5-b23b-ae971f0d2356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:59.350722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfde528-89b5-4ae5-b23b-ae971f0d2356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:59.351125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfde528-89b5-4ae5-b23b-ae971f0d2356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:59.351314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfde528-89b5-4ae5-b23b-ae971f0d2356 admin admin] PATH_INFO: `/v3/users/32e0c1804a26459184f8bca1c5ffdb8c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:59.352899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfde528-89b5-4ae5-b23b-ae971f0d2356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e0c1804a26459184f8bca1c5ffdb8c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:59.354949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfde528-89b5-4ae5-b23b-ae971f0d2356 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:59.357025 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:37:59.428317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 922/1843] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:37:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:37:59.450653 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed2b2b-84fc-4b6a-ab6f-aee279a6809c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:59.456076 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c6ed2b2b-84fc-4b6a-ab6f-aee279a6809c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:59.456646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8cfde528-89b5-4ae5-b23b-ae971f0d2356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e0c1804a26459184f8bca1c5ffdb8c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:59.470100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8cfde528-89b5-4ae5-b23b-ae971f0d2356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e0c1804a26459184f8bca1c5ffdb8c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:59.476327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed2b2b-84fc-4b6a-ab6f-aee279a6809c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:59.482400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df74d6e-708e-4bff-b959-37b421580548 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:59.483147 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df74d6e-708e-4bff-b959-37b421580548 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:59.483372 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df74d6e-708e-4bff-b959-37b421580548 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:59.483667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df74d6e-708e-4bff-b959-37b421580548 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:59.494413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df74d6e-708e-4bff-b959-37b421580548 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:59.496058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df74d6e-708e-4bff-b959-37b421580548 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:37:59.497984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8cfde528-89b5-4ae5-b23b-ae971f0d2356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e0c1804a26459184f8bca1c5ffdb8c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:59.498761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8cfde528-89b5-4ae5-b23b-ae971f0d2356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e0c1804a26459184f8bca1c5ffdb8c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:59.499443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8cfde528-89b5-4ae5-b23b-ae971f0d2356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e0c1804a26459184f8bca1c5ffdb8c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:37:59.510999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 923/1844] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:37:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:37:59.519082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 922/1845] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:37:59 2022] DELETE /identity/v3/users/32e0c1804a26459184f8bca1c5ffdb8c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:37:59.806806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df74d6e-708e-4bff-b959-37b421580548 service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:37:59.810662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3df74d6e-708e-4bff-b959-37b421580548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:37:59.831704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df74d6e-708e-4bff-b959-37b421580548 service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:37:59.913994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:37:59.914731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:37:59.915162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:37:59.915487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] PATH_INFO: `/v3/projects/35a657d0317b4d5b843c5b275aaffcca` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:37:59.916771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35a657d0317b4d5b843c5b275aaffcca)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:37:59.918968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:00.009023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35a657d0317b4d5b843c5b275aaffcca', 'request_id': 'req-f2101401-e9e3-45ad-b56a-41c708edf856'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:00.018791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35a657d0317b4d5b843c5b275aaffcca', 'request_id': 'req-f2101401-e9e3-45ad-b56a-41c708edf856'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:00.060413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35a657d0317b4d5b843c5b275aaffcca', 'request_id': 'req-f2101401-e9e3-45ad-b56a-41c708edf856'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:00.085385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] The token cache is being invalidate because project 35a657d0317b4d5b843c5b275aaffcca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:00.086201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:00.087265 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:00.094940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 924/1846] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:37:59 2022] DELETE /identity/v3/projects/35a657d0317b4d5b843c5b275aaffcca => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:00.100981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e8a57-b258-4419-8ed1-a2a53acc4404 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:00.100981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e8a57-b258-4419-8ed1-a2a53acc4404 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:00.101221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e8a57-b258-4419-8ed1-a2a53acc4404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:00.149924 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-947e8a57-b258-4419-8ed1-a2a53acc4404 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:00.151428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-947e8a57-b258-4419-8ed1-a2a53acc4404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:00.178839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:00.180546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:00.269927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:00.295205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 923/1847] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:00.298544 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc5d44-e4d9-46b8-b50b-9eb366e8730e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:00.298989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc5d44-e4d9-46b8-b50b-9eb366e8730e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:00.299427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc5d44-e4d9-46b8-b50b-9eb366e8730e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:00.321663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2101401-e9e3-45ad-b56a-41c708edf856 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:00.327643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c75791-bc5d-4120-ab22-b9349c34240d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:00.329257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c75791-bc5d-4120-ab22-b9349c34240d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:00.329970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c75791-bc5d-4120-ab22-b9349c34240d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:00.330447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c75791-bc5d-4120-ab22-b9349c34240d admin admin] PATH_INFO: `/v3/users/452c6f5043764780bf2533d69d79596d` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:00.332190 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c75791-bc5d-4120-ab22-b9349c34240d admin admin] RBAC: Authorizing `identity:delete_user(user_id=452c6f5043764780bf2533d69d79596d)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:00.334858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c75791-bc5d-4120-ab22-b9349c34240d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:00.342921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e3bc5d44-e4d9-46b8-b50b-9eb366e8730e None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:00.344294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e3bc5d44-e4d9-46b8-b50b-9eb366e8730e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:00.366680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 924/1848] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:00.369224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc5d44-e4d9-46b8-b50b-9eb366e8730e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:00.382483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc5d44-e4d9-46b8-b50b-9eb366e8730e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:00.389904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cab10-1a64-4f1a-bb9b-be70d495b388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:00.390519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cab10-1a64-4f1a-bb9b-be70d495b388 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:00.390799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cab10-1a64-4f1a-bb9b-be70d495b388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:00.394307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cab10-1a64-4f1a-bb9b-be70d495b388 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:00.396597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cab10-1a64-4f1a-bb9b-be70d495b388 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:00.396927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cab10-1a64-4f1a-bb9b-be70d495b388 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:00.398589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cab10-1a64-4f1a-bb9b-be70d495b388 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:00.412545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 925/1849] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:38:00.417136 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cab10-1a64-4f1a-bb9b-be70d495b388 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:00.432974 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cab10-1a64-4f1a-bb9b-be70d495b388 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:00.439899 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e3c75791-bc5d-4120-ab22-b9349c34240d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '452c6f5043764780bf2533d69d79596d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:00.453180 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0dd26-0ad0-419a-9c3a-52c4dc07fe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:00.454104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0dd26-0ad0-419a-9c3a-52c4dc07fe30 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:00.454583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0dd26-0ad0-419a-9c3a-52c4dc07fe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:00.455264 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0dd26-0ad0-419a-9c3a-52c4dc07fe30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:00.456336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0dd26-0ad0-419a-9c3a-52c4dc07fe30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:00.459393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0dd26-0ad0-419a-9c3a-52c4dc07fe30 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:00.482378 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e3c75791-bc5d-4120-ab22-b9349c34240d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '452c6f5043764780bf2533d69d79596d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:00.482378 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e3c75791-bc5d-4120-ab22-b9349c34240d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '452c6f5043764780bf2533d69d79596d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:00.487402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:00.498421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e3c75791-bc5d-4120-ab22-b9349c34240d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '452c6f5043764780bf2533d69d79596d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:00.499528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 926/1850] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:00 2022] POST /identity/v3/projects => generated 387 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:00.503959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0dd26-0ad0-419a-9c3a-52c4dc07fe30 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:00.508026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a0c0dd26-0ad0-419a-9c3a-52c4dc07fe30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:00.516170 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e3c75791-bc5d-4120-ab22-b9349c34240d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '452c6f5043764780bf2533d69d79596d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:00.525121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 925/1851] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:00 2022] DELETE /identity/v3/users/452c6f5043764780bf2533d69d79596d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:00.530343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138e8d1-14ad-4e1c-9114-e046779abe01 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:00.530635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138e8d1-14ad-4e1c-9114-e046779abe01 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:00.530916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138e8d1-14ad-4e1c-9114-e046779abe01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:00.567788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f138e8d1-14ad-4e1c-9114-e046779abe01 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:00.569107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f138e8d1-14ad-4e1c-9114-e046779abe01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:00.586483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0dd26-0ad0-419a-9c3a-52c4dc07fe30 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:00.659107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 926/1852] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:00.667951 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138e8d1-14ad-4e1c-9114-e046779abe01 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:00.669768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f138e8d1-14ad-4e1c-9114-e046779abe01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:00.716665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a495984-f939-4d9a-9a06-3a4362e6e4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:00.717403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a495984-f939-4d9a-9a06-3a4362e6e4bc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:00.717783 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a495984-f939-4d9a-9a06-3a4362e6e4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:00.718321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a495984-f939-4d9a-9a06-3a4362e6e4bc admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:00.719242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a495984-f939-4d9a-9a06-3a4362e6e4bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:00.721752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a495984-f939-4d9a-9a06-3a4362e6e4bc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:00.759679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 927/1853] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:00 2022] POST /identity/v3/users => generated 294 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:38:00.763811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a495984-f939-4d9a-9a06-3a4362e6e4bc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:00.779379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a495984-f939-4d9a-9a06-3a4362e6e4bc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:00.786589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b62f9d-98ca-443c-a85b-6e220e7eb241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:00.787351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b62f9d-98ca-443c-a85b-6e220e7eb241 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:00.787678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b62f9d-98ca-443c-a85b-6e220e7eb241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:00.788026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b62f9d-98ca-443c-a85b-6e220e7eb241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:00.788882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b62f9d-98ca-443c-a85b-6e220e7eb241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:00.790743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b62f9d-98ca-443c-a85b-6e220e7eb241 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:00.795365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138e8d1-14ad-4e1c-9114-e046779abe01 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:00.798975 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38359898-a45f-4c3b-9ff4-276aeaf9420a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:00.799490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38359898-a45f-4c3b-9ff4-276aeaf9420a service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:00.799776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38359898-a45f-4c3b-9ff4-276aeaf9420a service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:00.800056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38359898-a45f-4c3b-9ff4-276aeaf9420a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:00.807758 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 928/1854] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:38:00.811746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b62f9d-98ca-443c-a85b-6e220e7eb241 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:00.812571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38359898-a45f-4c3b-9ff4-276aeaf9420a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:00.814374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38359898-a45f-4c3b-9ff4-276aeaf9420a service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:00.822632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b62f9d-98ca-443c-a85b-6e220e7eb241 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:00.826463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 927/1855] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:38:00.828272 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5a44c-a0fb-47a5-831b-6816b550a010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:00.829069 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5a44c-a0fb-47a5-831b-6816b550a010 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:00.829566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5a44c-a0fb-47a5-831b-6816b550a010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:00.829837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5a44c-a0fb-47a5-831b-6816b550a010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:00.830673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5a44c-a0fb-47a5-831b-6816b550a010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:00.833038 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5a44c-a0fb-47a5-831b-6816b550a010 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:00.847131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 929/1856] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:38:00.851058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38359898-a45f-4c3b-9ff4-276aeaf9420a service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:00.858628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:00.865306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38359898-a45f-4c3b-9ff4-276aeaf9420a service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:00.876189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5720b-be26-4685-9829-e0dd2e4efad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:00.877271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5720b-be26-4685-9829-e0dd2e4efad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:00.877271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5720b-be26-4685-9829-e0dd2e4efad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:00.877558 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5720b-be26-4685-9829-e0dd2e4efad9 admin admin] PATH_INFO: `/v3/projects/b530dd4b1f2b4fc694aa7c6aa6887487/users/dc112b9ed65048d7ab9a3bd0d4179b61/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:00.904724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5720b-be26-4685-9829-e0dd2e4efad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b530dd4b1f2b4fc694aa7c6aa6887487, user_id=dc112b9ed65048d7ab9a3bd0d4179b61, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:00.922846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5720b-be26-4685-9829-e0dd2e4efad9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:00.987185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 928/1857] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:00 2022] PUT /identity/v3/projects/b530dd4b1f2b4fc694aa7c6aa6887487/users/dc112b9ed65048d7ab9a3bd0d4179b61/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:00.993148 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea53061-7f47-4e68-8480-1ec2a2d1e3d3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:00.993868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea53061-7f47-4e68-8480-1ec2a2d1e3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:00.993868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea53061-7f47-4e68-8480-1ec2a2d1e3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:01.048099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cea53061-7f47-4e68-8480-1ec2a2d1e3d3 None None] MFA Rules not processed for user `dc112b9ed65048d7ab9a3bd0d4179b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:01.049790 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cea53061-7f47-4e68-8480-1ec2a2d1e3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:01.176305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 930/1858] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:01.181342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5720b-be26-4685-9829-e0dd2e4efad9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:01.288055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5720b-be26-4685-9829-e0dd2e4efad9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:01.302643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231cefc-15e6-4d30-b06d-e7db61a2341c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:01.303325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231cefc-15e6-4d30-b06d-e7db61a2341c admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:01.303616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231cefc-15e6-4d30-b06d-e7db61a2341c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:01.303901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231cefc-15e6-4d30-b06d-e7db61a2341c admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:01.304592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231cefc-15e6-4d30-b06d-e7db61a2341c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:01.306709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231cefc-15e6-4d30-b06d-e7db61a2341c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:01.343783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 929/1859] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:01 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:01.348081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea53061-7f47-4e68-8480-1ec2a2d1e3d3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:01.355035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:01.361649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea53061-7f47-4e68-8480-1ec2a2d1e3d3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:01.445513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ae6eb-9345-41d8-8260-774893dc1604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:01.446254 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ae6eb-9345-41d8-8260-774893dc1604 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:01.446539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ae6eb-9345-41d8-8260-774893dc1604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:01.446816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ae6eb-9345-41d8-8260-774893dc1604 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:01.447773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ae6eb-9345-41d8-8260-774893dc1604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:01.451954 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ae6eb-9345-41d8-8260-774893dc1604 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:01.478338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 931/1860] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:01 2022] POST /identity/v3/users => generated 300 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:01.482332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231cefc-15e6-4d30-b06d-e7db61a2341c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:01.494630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231cefc-15e6-4d30-b06d-e7db61a2341c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:01.499891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654a6e0-a9ba-445d-8eff-71425df6cee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:01.500618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654a6e0-a9ba-445d-8eff-71425df6cee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:01.501002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654a6e0-a9ba-445d-8eff-71425df6cee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:01.501377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654a6e0-a9ba-445d-8eff-71425df6cee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:01.502954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654a6e0-a9ba-445d-8eff-71425df6cee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:01.505947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654a6e0-a9ba-445d-8eff-71425df6cee0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:01.522990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 930/1861] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:01.525527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ae6eb-9345-41d8-8260-774893dc1604 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:01.538884 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ae6eb-9345-41d8-8260-774893dc1604 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:01.543570 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2565936-79d1-4515-a1ed-3a86934e297d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:01.544252 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2565936-79d1-4515-a1ed-3a86934e297d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:01.544537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2565936-79d1-4515-a1ed-3a86934e297d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:01.544816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2565936-79d1-4515-a1ed-3a86934e297d admin admin] PATH_INFO: `/v3/projects/f36041d7e7fa4bb69175a3a072262848/users/9690ffce97f84507beadf97543d5e248/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:01.567756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2565936-79d1-4515-a1ed-3a86934e297d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36041d7e7fa4bb69175a3a072262848, user_id=9690ffce97f84507beadf97543d5e248, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:01.572195 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2565936-79d1-4515-a1ed-3a86934e297d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:01.592873 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 932/1862] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:01 2022] PUT /identity/v3/projects/f36041d7e7fa4bb69175a3a072262848/users/9690ffce97f84507beadf97543d5e248/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:01.597140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654a6e0-a9ba-445d-8eff-71425df6cee0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:01.610644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654a6e0-a9ba-445d-8eff-71425df6cee0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:01.658093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2565936-79d1-4515-a1ed-3a86934e297d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:01.660204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c2565936-79d1-4515-a1ed-3a86934e297d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:01.815747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a2e25-dd59-4381-a714-e585e949814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:01.816537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a2e25-dd59-4381-a714-e585e949814c admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:01.817091 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a2e25-dd59-4381-a714-e585e949814c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:01.817711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a2e25-dd59-4381-a714-e585e949814c admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:01.818600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a2e25-dd59-4381-a714-e585e949814c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:01.820364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a2e25-dd59-4381-a714-e585e949814c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:01.855628 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 931/1863] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:01.859808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a2e25-dd59-4381-a714-e585e949814c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:01.888447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a2e25-dd59-4381-a714-e585e949814c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:01.902487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afb638-e512-4322-9741-b56678776b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:01.902487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afb638-e512-4322-9741-b56678776b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:01.902487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afb638-e512-4322-9741-b56678776b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:01.902852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afb638-e512-4322-9741-b56678776b4e admin admin] PATH_INFO: `/v3/projects/f36041d7e7fa4bb69175a3a072262848/users/9690ffce97f84507beadf97543d5e248/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:01.906129 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afb638-e512-4322-9741-b56678776b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36041d7e7fa4bb69175a3a072262848, user_id=9690ffce97f84507beadf97543d5e248, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:01.908356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afb638-e512-4322-9741-b56678776b4e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:01.932699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2565936-79d1-4515-a1ed-3a86934e297d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:01.952056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 932/1864] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:01 2022] PUT /identity/v3/projects/f36041d7e7fa4bb69175a3a072262848/users/9690ffce97f84507beadf97543d5e248/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:01.963231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9382d0-589e-498a-8a71-a920f4f03a12 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:01.963718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9382d0-589e-498a-8a71-a920f4f03a12 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:01.964231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9382d0-589e-498a-8a71-a920f4f03a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:02.046626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-cb9382d0-589e-498a-8a71-a920f4f03a12 None None] MFA Rules not processed for user `9690ffce97f84507beadf97543d5e248`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:02.049553 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cb9382d0-589e-498a-8a71-a920f4f03a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:02.087033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:02.087725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:02.088137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:02.088457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] PATH_INFO: `/v3/projects/7501231f33de47bc8c851baa8bde1174` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:02.091003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7501231f33de47bc8c851baa8bde1174)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:02.096203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:02.112407 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:02.143191 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:02.203841 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7501231f33de47bc8c851baa8bde1174', 'request_id': 'req-0167f652-4e87-405f-bb53-a41a941cbfe1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:02.218092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7501231f33de47bc8c851baa8bde1174', 'request_id': 'req-0167f652-4e87-405f-bb53-a41a941cbfe1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:02.231748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 933/1865] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:02.233421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7501231f33de47bc8c851baa8bde1174', 'request_id': 'req-0167f652-4e87-405f-bb53-a41a941cbfe1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:02.240639 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9382d0-589e-498a-8a71-a920f4f03a12 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:02.242077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cb9382d0-589e-498a-8a71-a920f4f03a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:02.264893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] The token cache is being invalidate because project 7501231f33de47bc8c851baa8bde1174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:02.265265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:02.266654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:02.271564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 933/1866] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:01 2022] DELETE /identity/v3/projects/7501231f33de47bc8c851baa8bde1174 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:38:02.275233 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:02.279590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:02.384313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9382d0-589e-498a-8a71-a920f4f03a12 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:02.434859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167f652-4e87-405f-bb53-a41a941cbfe1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:02.449965 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230411d-a1d7-4d61-8e7e-a584955d1010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:02.451009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230411d-a1d7-4d61-8e7e-a584955d1010 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:02.451148 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9c927-f3d7-4788-bed0-a5f42fa5906b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:02.451642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230411d-a1d7-4d61-8e7e-a584955d1010 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:02.451814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9c927-f3d7-4788-bed0-a5f42fa5906b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:02.452220 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9c927-f3d7-4788-bed0-a5f42fa5906b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:02.452579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9c927-f3d7-4788-bed0-a5f42fa5906b admin admin] PATH_INFO: `/v3/users/fb26172ff1524b9897bb13afde9166a0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:02.452850 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230411d-a1d7-4d61-8e7e-a584955d1010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:02.453935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9c927-f3d7-4788-bed0-a5f42fa5906b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb26172ff1524b9897bb13afde9166a0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:02.455402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0230411d-a1d7-4d61-8e7e-a584955d1010 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:02.456081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9c927-f3d7-4788-bed0-a5f42fa5906b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:02.539880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-79b9c927-f3d7-4788-bed0-a5f42fa5906b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb26172ff1524b9897bb13afde9166a0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:02.554072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-79b9c927-f3d7-4788-bed0-a5f42fa5906b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb26172ff1524b9897bb13afde9166a0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:02.574490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230411d-a1d7-4d61-8e7e-a584955d1010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:02.574841 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-79b9c927-f3d7-4788-bed0-a5f42fa5906b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb26172ff1524b9897bb13afde9166a0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:02.575701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-79b9c927-f3d7-4788-bed0-a5f42fa5906b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb26172ff1524b9897bb13afde9166a0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:02.575997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230411d-a1d7-4d61-8e7e-a584955d1010 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:02.576464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-79b9c927-f3d7-4788-bed0-a5f42fa5906b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb26172ff1524b9897bb13afde9166a0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:02.586197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 934/1867] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:38:02.589758 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 934/1868] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:02 2022] DELETE /identity/v3/users/fb26172ff1524b9897bb13afde9166a0 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:02.692453 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230411d-a1d7-4d61-8e7e-a584955d1010 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:02.694803 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0230411d-a1d7-4d61-8e7e-a584955d1010 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:02.720609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230411d-a1d7-4d61-8e7e-a584955d1010 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:02.787491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:02.829916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dfd8a-3b73-44b8-bb36-1f1f42ff19dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:02.830599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dfd8a-3b73-44b8-bb36-1f1f42ff19dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:02.831321 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dfd8a-3b73-44b8-bb36-1f1f42ff19dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:02.831831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dfd8a-3b73-44b8-bb36-1f1f42ff19dd admin admin] PATH_INFO: `/v3/users/dc112b9ed65048d7ab9a3bd0d4179b61` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:02.833976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dfd8a-3b73-44b8-bb36-1f1f42ff19dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc112b9ed65048d7ab9a3bd0d4179b61)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:02.836982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dfd8a-3b73-44b8-bb36-1f1f42ff19dd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:02.974838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-832dfd8a-3b73-44b8-bb36-1f1f42ff19dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc112b9ed65048d7ab9a3bd0d4179b61'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:03.001066 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-832dfd8a-3b73-44b8-bb36-1f1f42ff19dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc112b9ed65048d7ab9a3bd0d4179b61'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:03.001968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-832dfd8a-3b73-44b8-bb36-1f1f42ff19dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc112b9ed65048d7ab9a3bd0d4179b61'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:03.014370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-832dfd8a-3b73-44b8-bb36-1f1f42ff19dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc112b9ed65048d7ab9a3bd0d4179b61'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:03.028602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-832dfd8a-3b73-44b8-bb36-1f1f42ff19dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc112b9ed65048d7ab9a3bd0d4179b61'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:03.039924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 935/1869] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:02 2022] DELETE /identity/v3/users/dc112b9ed65048d7ab9a3bd0d4179b61 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:03.044294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94dc5c-30e5-49bc-8bbd-ccba2d8a505c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:03.044591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94dc5c-30e5-49bc-8bbd-ccba2d8a505c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:03.044879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94dc5c-30e5-49bc-8bbd-ccba2d8a505c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:03.073521 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-da94dc5c-30e5-49bc-8bbd-ccba2d8a505c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:03.075727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-da94dc5c-30e5-49bc-8bbd-ccba2d8a505c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:03.109167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:03.149456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b013a-11b5-4bb9-9fc7-15582070b29f None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:03.149938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b013a-11b5-4bb9-9fc7-15582070b29f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:03.150362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b013a-11b5-4bb9-9fc7-15582070b29f None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:03.152388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 936/1870] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:38:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:38:03.189545 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b013a-11b5-4bb9-9fc7-15582070b29f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:03.192359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e94b013a-11b5-4bb9-9fc7-15582070b29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:03.241482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 935/1871] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:03.242797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276530ec-e8d7-4f6f-ba21-6e8a49fb1808 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:03.242797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276530ec-e8d7-4f6f-ba21-6e8a49fb1808 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:03.242797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276530ec-e8d7-4f6f-ba21-6e8a49fb1808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:03.287160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-276530ec-e8d7-4f6f-ba21-6e8a49fb1808 None None] MFA Rules not processed for user `928c5d1284b74f1291c031323bb03a20`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:03.288887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-276530ec-e8d7-4f6f-ba21-6e8a49fb1808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:03.289479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b013a-11b5-4bb9-9fc7-15582070b29f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:03.303166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:03.304042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:03.304340 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:03.304690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d admin admin] PATH_INFO: `/v3/projects/e81a7548322343feb188cd72fa6cace9` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:03.306059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81a7548322343feb188cd72fa6cace9)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:03.309859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:03.382941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 936/1872] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:38:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:03.383948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276530ec-e8d7-4f6f-ba21-6e8a49fb1808 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:03.385437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-276530ec-e8d7-4f6f-ba21-6e8a49fb1808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:03.395632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81a7548322343feb188cd72fa6cace9', 'request_id': 'req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:03.397287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:03.407676 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81a7548322343feb188cd72fa6cace9', 'request_id': 'req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:03.416072 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81a7548322343feb188cd72fa6cace9', 'request_id': 'req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:03.432392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d admin admin] The token cache is being invalidate because project e81a7548322343feb188cd72fa6cace9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:03.432934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:03.434132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f822ca69-48c7-4f60-b6ff-59bfa1c4907d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:03.436538 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 937/1873] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:03 2022] DELETE /identity/v3/projects/e81a7548322343feb188cd72fa6cace9 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:03.441641 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23185207-b296-4716-b73e-6c205b766cb5 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:03.444194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23185207-b296-4716-b73e-6c205b766cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:03.445071 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23185207-b296-4716-b73e-6c205b766cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:03.458658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276530ec-e8d7-4f6f-ba21-6e8a49fb1808 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:03.502571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-23185207-b296-4716-b73e-6c205b766cb5 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:03.503847 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-23185207-b296-4716-b73e-6c205b766cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:03.572375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721adc82-bfe7-4622-b5cd-0573bf481a5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:03.578832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721adc82-bfe7-4622-b5cd-0573bf481a5a service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:03.579949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721adc82-bfe7-4622-b5cd-0573bf481a5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:03.581139 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721adc82-bfe7-4622-b5cd-0573bf481a5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:03.585214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-721adc82-bfe7-4622-b5cd-0573bf481a5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:03.635707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 938/1874] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:03.635707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23185207-b296-4716-b73e-6c205b766cb5 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:03.635707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-23185207-b296-4716-b73e-6c205b766cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:03.707814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:03.767567 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721adc82-bfe7-4622-b5cd-0573bf481a5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:03.770977 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23185207-b296-4716-b73e-6c205b766cb5 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:03.774270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721adc82-bfe7-4622-b5cd-0573bf481a5a service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:03.778086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79e7ee-fead-4c14-a638-7cd35aeb1db0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928c5d1284b74f1291c031323bb03a20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:03.779329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79e7ee-fead-4c14-a638-7cd35aeb1db0 service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:03.779788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79e7ee-fead-4c14-a638-7cd35aeb1db0 service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:03.780233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79e7ee-fead-4c14-a638-7cd35aeb1db0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:03.782482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2f79e7ee-fead-4c14-a638-7cd35aeb1db0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:03.798620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79e7ee-fead-4c14-a638-7cd35aeb1db0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:03.801519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79e7ee-fead-4c14-a638-7cd35aeb1db0 service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:03.803365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 937/1875] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:03.806508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2c5a0-cfe6-428e-839b-39f62b78e7d2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:03.807413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2c5a0-cfe6-428e-839b-39f62b78e7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:03.809333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2c5a0-cfe6-428e-839b-39f62b78e7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:03.816793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 939/1876] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:03.848811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8db2c5a0-cfe6-428e-839b-39f62b78e7d2 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:03.850868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8db2c5a0-cfe6-428e-839b-39f62b78e7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:03.861573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 938/1877] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:03.867832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79e7ee-fead-4c14-a638-7cd35aeb1db0 service placement] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:03.887956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79e7ee-fead-4c14-a638-7cd35aeb1db0 service placement] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:03.894107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7de700-452d-48e4-8fff-2b8f623c55d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:03.895636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7de700-452d-48e4-8fff-2b8f623c55d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:03.896330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7de700-452d-48e4-8fff-2b8f623c55d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:03.897095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7de700-452d-48e4-8fff-2b8f623c55d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:03.898340 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7de700-452d-48e4-8fff-2b8f623c55d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:03.898852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7de700-452d-48e4-8fff-2b8f623c55d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:03.901434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7de700-452d-48e4-8fff-2b8f623c55d5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:03.926758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 940/1878] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:38:03.927428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2c5a0-cfe6-428e-839b-39f62b78e7d2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:03.952239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2c5a0-cfe6-428e-839b-39f62b78e7d2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:03.983915 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548f2f4-3835-4912-bcdd-8f8c1e979925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:03.984935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548f2f4-3835-4912-bcdd-8f8c1e979925 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:03.985378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548f2f4-3835-4912-bcdd-8f8c1e979925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:03.985720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548f2f4-3835-4912-bcdd-8f8c1e979925 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:03.988264 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548f2f4-3835-4912-bcdd-8f8c1e979925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:03.996118 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548f2f4-3835-4912-bcdd-8f8c1e979925 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:04.066162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7de700-452d-48e4-8fff-2b8f623c55d5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:04.076273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 939/1879] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:03 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:38:04.079391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7de700-452d-48e4-8fff-2b8f623c55d5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:04.178437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:04.200016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf93150-e960-4f1e-ae31-9ddc1f960f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:04.201936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf93150-e960-4f1e-ae31-9ddc1f960f68 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:04.202567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf93150-e960-4f1e-ae31-9ddc1f960f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:04.203524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf93150-e960-4f1e-ae31-9ddc1f960f68 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:04.205782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf93150-e960-4f1e-ae31-9ddc1f960f68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:04.209547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf93150-e960-4f1e-ae31-9ddc1f960f68 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:04.246500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 941/1880] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:04 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:38:04.250719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548f2f4-3835-4912-bcdd-8f8c1e979925 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:04.262214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548f2f4-3835-4912-bcdd-8f8c1e979925 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:04.267274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5aefd-6628-402f-a579-4c918aebd79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:04.267893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5aefd-6628-402f-a579-4c918aebd79d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:04.268178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5aefd-6628-402f-a579-4c918aebd79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:04.268447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5aefd-6628-402f-a579-4c918aebd79d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:04.271807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5aefd-6628-402f-a579-4c918aebd79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:04.271807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5aefd-6628-402f-a579-4c918aebd79d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:04.286717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 940/1881] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:04.290881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf93150-e960-4f1e-ae31-9ddc1f960f68 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:04.301660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf93150-e960-4f1e-ae31-9ddc1f960f68 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:04.306609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97133b58-bb2c-4eda-928b-0a08f25c0518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:04.307358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97133b58-bb2c-4eda-928b-0a08f25c0518 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:04.307745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97133b58-bb2c-4eda-928b-0a08f25c0518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:04.308093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97133b58-bb2c-4eda-928b-0a08f25c0518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:04.308897 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97133b58-bb2c-4eda-928b-0a08f25c0518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:04.310589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97133b58-bb2c-4eda-928b-0a08f25c0518 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:04.322564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 942/1882] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:38:04.326716 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5aefd-6628-402f-a579-4c918aebd79d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:04.337014 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5aefd-6628-402f-a579-4c918aebd79d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:04.342173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a97e6d-c156-4178-a9ee-690daa793477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:04.342856 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a97e6d-c156-4178-a9ee-690daa793477 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:04.343170 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a97e6d-c156-4178-a9ee-690daa793477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:04.343450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a97e6d-c156-4178-a9ee-690daa793477 admin admin] PATH_INFO: `/v3/projects/a17ba96312b64eee99ab3d1e65d53807/users/f9970b66ff9d40f7a0753831a2d02ec7/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:04.360333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a97e6d-c156-4178-a9ee-690daa793477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a17ba96312b64eee99ab3d1e65d53807, user_id=f9970b66ff9d40f7a0753831a2d02ec7, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:04.362923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a97e6d-c156-4178-a9ee-690daa793477 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:04.378681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 941/1883] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:04 2022] PUT /identity/v3/projects/a17ba96312b64eee99ab3d1e65d53807/users/f9970b66ff9d40f7a0753831a2d02ec7/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:04.389950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1ebb7-aed3-45b7-974b-d2e0d532bfbe None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:04.390253 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1ebb7-aed3-45b7-974b-d2e0d532bfbe None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:04.390527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1ebb7-aed3-45b7-974b-d2e0d532bfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:04.422322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-34d1ebb7-aed3-45b7-974b-d2e0d532bfbe None None] MFA Rules not processed for user `f9970b66ff9d40f7a0753831a2d02ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:04.423697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-34d1ebb7-aed3-45b7-974b-d2e0d532bfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:04.472104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a97e6d-c156-4178-a9ee-690daa793477 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:04.474858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-10a97e6d-c156-4178-a9ee-690daa793477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:04.567437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 943/1884] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:04.579715 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1ebb7-aed3-45b7-974b-d2e0d532bfbe None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:04.581276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-34d1ebb7-aed3-45b7-974b-d2e0d532bfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:04.612432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a97e6d-c156-4178-a9ee-690daa793477 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:04.620913 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:04.621960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:04.622456 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:04.622949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] PATH_INFO: `/v3/projects/b530dd4b1f2b4fc694aa7c6aa6887487` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:04.627142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b530dd4b1f2b4fc694aa7c6aa6887487)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:04.646114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:04.753286 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1ebb7-aed3-45b7-974b-d2e0d532bfbe None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:04.797152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b530dd4b1f2b4fc694aa7c6aa6887487', 'request_id': 'req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:04.820741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b530dd4b1f2b4fc694aa7c6aa6887487', 'request_id': 'req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:04.836757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b530dd4b1f2b4fc694aa7c6aa6887487', 'request_id': 'req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:04.840323 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:04.856172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] The token cache is being invalidate because project b530dd4b1f2b4fc694aa7c6aa6887487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:04.856783 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:04.858154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:04.860884 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 942/1885] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:04 2022] DELETE /identity/v3/projects/b530dd4b1f2b4fc694aa7c6aa6887487 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:04.865035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:04.866735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:04.880859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e423b3-8967-4445-b32b-916c7d6f0a85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:04.881554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e423b3-8967-4445-b32b-916c7d6f0a85 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:04.881876 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e423b3-8967-4445-b32b-916c7d6f0a85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:04.882216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e423b3-8967-4445-b32b-916c7d6f0a85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:04.884517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-00e423b3-8967-4445-b32b-916c7d6f0a85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:04.923494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:04.956257 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9593c-edbd-46c8-bcc7-b53f3a19bd25 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:04.960693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e423b3-8967-4445-b32b-916c7d6f0a85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:04.962521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e423b3-8967-4445-b32b-916c7d6f0a85 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:04.962658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa35edb-6ffa-4a57-8c1e-e44c86d0ff67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:04.963438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa35edb-6ffa-4a57-8c1e-e44c86d0ff67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:04.963759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa35edb-6ffa-4a57-8c1e-e44c86d0ff67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:04.964051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa35edb-6ffa-4a57-8c1e-e44c86d0ff67 admin admin] PATH_INFO: `/v3/users/9690ffce97f84507beadf97543d5e248` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:04.967825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa35edb-6ffa-4a57-8c1e-e44c86d0ff67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9690ffce97f84507beadf97543d5e248)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:04.969655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa35edb-6ffa-4a57-8c1e-e44c86d0ff67 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:04.991457 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 944/1886] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:04.995229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df314405-6bb9-4fdf-b458-bd29c7a7a67b None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:04.995379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df314405-6bb9-4fdf-b458-bd29c7a7a67b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:04.995572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df314405-6bb9-4fdf-b458-bd29c7a7a67b None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:05.000256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 945/1887] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:38:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:38:05.038753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7fa35edb-6ffa-4a57-8c1e-e44c86d0ff67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9690ffce97f84507beadf97543d5e248'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:05.047165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df314405-6bb9-4fdf-b458-bd29c7a7a67b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:05.049300 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-df314405-6bb9-4fdf-b458-bd29c7a7a67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:05.063772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7fa35edb-6ffa-4a57-8c1e-e44c86d0ff67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9690ffce97f84507beadf97543d5e248'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:05.095024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df314405-6bb9-4fdf-b458-bd29c7a7a67b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:05.110083 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:05.113281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7fa35edb-6ffa-4a57-8c1e-e44c86d0ff67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9690ffce97f84507beadf97543d5e248'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:05.115362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7fa35edb-6ffa-4a57-8c1e-e44c86d0ff67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9690ffce97f84507beadf97543d5e248'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:05.116019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7fa35edb-6ffa-4a57-8c1e-e44c86d0ff67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9690ffce97f84507beadf97543d5e248'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:05.126650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:05.127111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:05.127390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] PATH_INFO: `/v3/users/1513d735a9b040c1b2cddb9a470d3e95` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:05.131647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1513d735a9b040c1b2cddb9a470d3e95)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:05.150076 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:05.183512 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 943/1888] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:04 2022] DELETE /identity/v3/users/9690ffce97f84507beadf97543d5e248 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:05.190440 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2336222-fbe8-4976-8655-1c3de64ac3b2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:05.190732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2336222-fbe8-4976-8655-1c3de64ac3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:05.191014 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2336222-fbe8-4976-8655-1c3de64ac3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:05.269546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e2336222-fbe8-4976-8655-1c3de64ac3b2 None None] MFA Rules not processed for user `928c5d1284b74f1291c031323bb03a20`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:05.274169 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e2336222-fbe8-4976-8655-1c3de64ac3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:05.315183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1513d735a9b040c1b2cddb9a470d3e95'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:05.364464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1513d735a9b040c1b2cddb9a470d3e95'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:05.375711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1513d735a9b040c1b2cddb9a470d3e95'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:05.391841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1513d735a9b040c1b2cddb9a470d3e95'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:05.425121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1513d735a9b040c1b2cddb9a470d3e95'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:05.435871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 946/1889] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:05 2022] DELETE /identity/v3/users/1513d735a9b040c1b2cddb9a470d3e95 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:38:05.509055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 944/1890] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:38:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:05.536094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:05.539204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:05.641527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8a704-320f-424b-99d7-8af9e2b90484 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:05.650411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b7b37-c105-4551-858e-115687739c59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928c5d1284b74f1291c031323bb03a20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:05.651413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b7b37-c105-4551-858e-115687739c59 service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:05.651923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b7b37-c105-4551-858e-115687739c59 service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:05.652532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b7b37-c105-4551-858e-115687739c59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:05.664591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b7b37-c105-4551-858e-115687739c59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:05.666843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b7b37-c105-4551-858e-115687739c59 service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:05.683377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 947/1891] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:05.924816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2336222-fbe8-4976-8655-1c3de64ac3b2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:05.927352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e2336222-fbe8-4976-8655-1c3de64ac3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:05.933668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b7b37-c105-4551-858e-115687739c59 service placement] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:05.935183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-629b7b37-c105-4551-858e-115687739c59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:05.970089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:06.021905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:06.064752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b7b37-c105-4551-858e-115687739c59 service placement] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:06.069922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:06.070608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:06.070896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:06.071210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] PATH_INFO: `/v3/projects/a31ee41299df4ab3a0c40ab6997eb3da` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:06.072436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a31ee41299df4ab3a0c40ab6997eb3da)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:06.074400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:06.079663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2336222-fbe8-4976-8655-1c3de64ac3b2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:06.084909 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42021a-951f-4873-b6ee-701fdea10b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:06.085630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42021a-951f-4873-b6ee-701fdea10b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:06.085985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42021a-951f-4873-b6ee-701fdea10b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:06.086333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42021a-951f-4873-b6ee-701fdea10b3c admin admin] PATH_INFO: `/v3/projects/f36041d7e7fa4bb69175a3a072262848` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:06.087604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42021a-951f-4873-b6ee-701fdea10b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f36041d7e7fa4bb69175a3a072262848)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:06.089600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42021a-951f-4873-b6ee-701fdea10b3c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:06.133013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a31ee41299df4ab3a0c40ab6997eb3da', 'request_id': 'req-e04d5d89-04b8-4669-a0ce-6565835a94a6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.139291 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a31ee41299df4ab3a0c40ab6997eb3da', 'request_id': 'req-e04d5d89-04b8-4669-a0ce-6565835a94a6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.147016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a31ee41299df4ab3a0c40ab6997eb3da', 'request_id': 'req-e04d5d89-04b8-4669-a0ce-6565835a94a6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.147611 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5f42021a-951f-4873-b6ee-701fdea10b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36041d7e7fa4bb69175a3a072262848', 'request_id': 'req-5f42021a-951f-4873-b6ee-701fdea10b3c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.158889 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5f42021a-951f-4873-b6ee-701fdea10b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36041d7e7fa4bb69175a3a072262848', 'request_id': 'req-5f42021a-951f-4873-b6ee-701fdea10b3c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.161990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] The token cache is being invalidate because project a31ee41299df4ab3a0c40ab6997eb3da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:06.162294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.162984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.167582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 948/1892] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:05 2022] DELETE /identity/v3/projects/a31ee41299df4ab3a0c40ab6997eb3da => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:06.168601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:06.169071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5f42021a-951f-4873-b6ee-701fdea10b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36041d7e7fa4bb69175a3a072262848', 'request_id': 'req-5f42021a-951f-4873-b6ee-701fdea10b3c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.170787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:06.182253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5f42021a-951f-4873-b6ee-701fdea10b3c admin admin] The token cache is being invalidate because project f36041d7e7fa4bb69175a3a072262848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:06.182558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5f42021a-951f-4873-b6ee-701fdea10b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.183289 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5f42021a-951f-4873-b6ee-701fdea10b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.185095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 945/1893] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:05 2022] DELETE /identity/v3/projects/f36041d7e7fa4bb69175a3a072262848 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:06.190957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df1957-ce9c-46d5-a738-bdb33587ac97 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:06.190957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df1957-ce9c-46d5-a738-bdb33587ac97 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:06.191398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df1957-ce9c-46d5-a738-bdb33587ac97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:06.212742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-15df1957-ce9c-46d5-a738-bdb33587ac97 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:06.213967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-15df1957-ce9c-46d5-a738-bdb33587ac97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:06.246705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d5d89-04b8-4669-a0ce-6565835a94a6 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:06.283368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 946/1894] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:06.290577 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cd453-99a7-43eb-8dd3-056e29b3391a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:06.290869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cd453-99a7-43eb-8dd3-056e29b3391a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:06.291192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cd453-99a7-43eb-8dd3-056e29b3391a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:06.311415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-961cd453-99a7-43eb-8dd3-056e29b3391a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:06.312618 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-961cd453-99a7-43eb-8dd3-056e29b3391a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:06.314242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:06.315137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:06.315427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:06.315999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] PATH_INFO: `/v3/users/8d027f219bb345dca37c0ae3d9c0d8fb` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:06.317667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d027f219bb345dca37c0ae3d9c0d8fb)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:06.319695 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:06.324846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 947/1895] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:06.326430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cd453-99a7-43eb-8dd3-056e29b3391a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:06.336873 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cd453-99a7-43eb-8dd3-056e29b3391a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:06.338730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:06.345037 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1cb4a-103f-42d3-86ee-bf970779ee2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:06.346129 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1cb4a-103f-42d3-86ee-bf970779ee2d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:06.346580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1cb4a-103f-42d3-86ee-bf970779ee2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:06.347010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1cb4a-103f-42d3-86ee-bf970779ee2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:06.347934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1cb4a-103f-42d3-86ee-bf970779ee2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:06.348391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1cb4a-103f-42d3-86ee-bf970779ee2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:06.351119 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1cb4a-103f-42d3-86ee-bf970779ee2d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:06.369857 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 948/1896] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:38:06.370996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1cb4a-103f-42d3-86ee-bf970779ee2d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:06.388134 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1cb4a-103f-42d3-86ee-bf970779ee2d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:06.398019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d872fd-bf8d-48ea-91cc-ea38e6d41da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:06.398892 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d872fd-bf8d-48ea-91cc-ea38e6d41da2 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:06.399376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d872fd-bf8d-48ea-91cc-ea38e6d41da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:06.400058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d872fd-bf8d-48ea-91cc-ea38e6d41da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:06.400964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d872fd-bf8d-48ea-91cc-ea38e6d41da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:06.403141 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d872fd-bf8d-48ea-91cc-ea38e6d41da2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:06.415631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d027f219bb345dca37c0ae3d9c0d8fb'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.431277 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d027f219bb345dca37c0ae3d9c0d8fb'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.432121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d027f219bb345dca37c0ae3d9c0d8fb'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.440142 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d027f219bb345dca37c0ae3d9c0d8fb'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.457392 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:06.464163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d027f219bb345dca37c0ae3d9c0d8fb'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:06.469029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 949/1897] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:06 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:06.469799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d872fd-bf8d-48ea-91cc-ea38e6d41da2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:06.474149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-19d872fd-bf8d-48ea-91cc-ea38e6d41da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:06.503203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 949/1898] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:06 2022] DELETE /identity/v3/users/8d027f219bb345dca37c0ae3d9c0d8fb => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:06.704598 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d872fd-bf8d-48ea-91cc-ea38e6d41da2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:06.808448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d40630-379d-4998-aed6-713ea7e637ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:06.809500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d40630-379d-4998-aed6-713ea7e637ba admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:06.809963 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d40630-379d-4998-aed6-713ea7e637ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:06.810298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d40630-379d-4998-aed6-713ea7e637ba admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:06.811102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d40630-379d-4998-aed6-713ea7e637ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:06.814270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d40630-379d-4998-aed6-713ea7e637ba admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:06.875163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 950/1899] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:06 2022] POST /identity/v3/users => generated 289 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:06.875820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:06.905045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426ec55-9cf8-4e4d-aed3-22bd87f2c4a6 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:06.929729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328a98-e96e-45b1-80d9-9ac952bd73ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:06.930381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328a98-e96e-45b1-80d9-9ac952bd73ab admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:06.930674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328a98-e96e-45b1-80d9-9ac952bd73ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:06.930958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328a98-e96e-45b1-80d9-9ac952bd73ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:06.931829 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e328a98-e96e-45b1-80d9-9ac952bd73ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:06.933727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e328a98-e96e-45b1-80d9-9ac952bd73ab admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:06.962471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 950/1900] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:38:06.963718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d40630-379d-4998-aed6-713ea7e637ba admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:06.978411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d40630-379d-4998-aed6-713ea7e637ba admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:06.983224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1cc70-0117-4b9f-81ac-394ffe96779a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:06.983893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1cc70-0117-4b9f-81ac-394ffe96779a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:06.984177 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1cc70-0117-4b9f-81ac-394ffe96779a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:06.984455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1cc70-0117-4b9f-81ac-394ffe96779a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:06.985165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1cc70-0117-4b9f-81ac-394ffe96779a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:06.987223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1cc70-0117-4b9f-81ac-394ffe96779a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:07.000648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 951/1901] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:07.007659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328a98-e96e-45b1-80d9-9ac952bd73ab admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:07.027593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328a98-e96e-45b1-80d9-9ac952bd73ab admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:07.033602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ab4d9-efbe-4228-befb-ceda8e3b4cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:07.034447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ab4d9-efbe-4228-befb-ceda8e3b4cce admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.034833 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ab4d9-efbe-4228-befb-ceda8e3b4cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.035921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ab4d9-efbe-4228-befb-ceda8e3b4cce admin admin] PATH_INFO: `/v3/projects/63fb4664783f4667ba10d567e5c0c5be/users/bdd92e8b7be74f8696b881dc20e2aabf/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.043792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1cc70-0117-4b9f-81ac-394ffe96779a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:07.050525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-70f1cc70-0117-4b9f-81ac-394ffe96779a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:07.070757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1cc70-0117-4b9f-81ac-394ffe96779a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:07.084070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ab4d9-efbe-4228-befb-ceda8e3b4cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=63fb4664783f4667ba10d567e5c0c5be, user_id=bdd92e8b7be74f8696b881dc20e2aabf, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:07.092524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ab4d9-efbe-4228-befb-ceda8e3b4cce admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:07.096647 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:07.097318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.098907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.099244 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e admin admin] PATH_INFO: `/v3/projects/6100c8af2ca44801b2b6b84ac3f18b18` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.100816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6100c8af2ca44801b2b6b84ac3f18b18)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:07.104481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:07.124132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 951/1902] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:07 2022] PUT /identity/v3/projects/63fb4664783f4667ba10d567e5c0c5be/users/bdd92e8b7be74f8696b881dc20e2aabf/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:07.138234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ff397-6b85-49a8-9e5f-97afa9b2c47a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.138804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ff397-6b85-49a8-9e5f-97afa9b2c47a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.139253 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ff397-6b85-49a8-9e5f-97afa9b2c47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.274859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b65ff397-6b85-49a8-9e5f-97afa9b2c47a None None] MFA Rules not processed for user `bdd92e8b7be74f8696b881dc20e2aabf`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:07.275819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6100c8af2ca44801b2b6b84ac3f18b18', 'request_id': 'req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:07.276498 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b65ff397-6b85-49a8-9e5f-97afa9b2c47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:07.297562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6100c8af2ca44801b2b6b84ac3f18b18', 'request_id': 'req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:07.306514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6100c8af2ca44801b2b6b84ac3f18b18', 'request_id': 'req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:07.319057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e admin admin] The token cache is being invalidate because project 6100c8af2ca44801b2b6b84ac3f18b18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:07.319375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:07.320205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-be5e5f30-1abd-4d8b-a694-aafe0eb2e35e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:07.322319 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 952/1903] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:07 2022] DELETE /identity/v3/projects/6100c8af2ca44801b2b6b84ac3f18b18 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:07.326950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a7839-e659-4d3e-9ccc-042ee154857e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.327263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a7839-e659-4d3e-9ccc-042ee154857e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.327537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a7839-e659-4d3e-9ccc-042ee154857e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.350078 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-be7a7839-e659-4d3e-9ccc-042ee154857e None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:07.351057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-be7a7839-e659-4d3e-9ccc-042ee154857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:07.355558 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 952/1904] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:07.361084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:07.366946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ff397-6b85-49a8-9e5f-97afa9b2c47a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:07.368944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b65ff397-6b85-49a8-9e5f-97afa9b2c47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:07.424126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 953/1905] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:07.433310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:07.433518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e9405-0f8b-4bfe-bf54-94a4186eca11 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.433776 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e9405-0f8b-4bfe-bf54-94a4186eca11 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.434055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e9405-0f8b-4bfe-bf54-94a4186eca11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.440182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ff397-6b85-49a8-9e5f-97afa9b2c47a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:07.444703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa89c4-6c67-4c19-bfd6-2bbeb3dfae7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:07.446588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa89c4-6c67-4c19-bfd6-2bbeb3dfae7f service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.446924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa89c4-6c67-4c19-bfd6-2bbeb3dfae7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.447484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa89c4-6c67-4c19-bfd6-2bbeb3dfae7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.456178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-443e9405-0f8b-4bfe-bf54-94a4186eca11 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:07.458059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-443e9405-0f8b-4bfe-bf54-94a4186eca11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:07.458215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa89c4-6c67-4c19-bfd6-2bbeb3dfae7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:07.459799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa89c4-6c67-4c19-bfd6-2bbeb3dfae7f service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:07.469626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 954/1906] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:07.470098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 953/1907] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:07.471643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e9405-0f8b-4bfe-bf54-94a4186eca11 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:07.483587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e9405-0f8b-4bfe-bf54-94a4186eca11 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:07.489984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64fefd-04d4-463d-bd37-832a005b9c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:07.490528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64fefd-04d4-463d-bd37-832a005b9c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.491499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64fefd-04d4-463d-bd37-832a005b9c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.491945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64fefd-04d4-463d-bd37-832a005b9c8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.492746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64fefd-04d4-463d-bd37-832a005b9c8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:07.493133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64fefd-04d4-463d-bd37-832a005b9c8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:07.494936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64fefd-04d4-463d-bd37-832a005b9c8c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:07.511111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 955/1908] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:38:07.513284 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa89c4-6c67-4c19-bfd6-2bbeb3dfae7f service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:07.526332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa89c4-6c67-4c19-bfd6-2bbeb3dfae7f service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:07.532262 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01376cb-3b57-46a6-84ec-b698c57594e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:07.533218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01376cb-3b57-46a6-84ec-b698c57594e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.534265 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01376cb-3b57-46a6-84ec-b698c57594e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.534723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01376cb-3b57-46a6-84ec-b698c57594e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.537109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01376cb-3b57-46a6-84ec-b698c57594e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:07.540298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01376cb-3b57-46a6-84ec-b698c57594e9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:07.579511 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 954/1909] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:07 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:07.583840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64fefd-04d4-463d-bd37-832a005b9c8c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:07.594917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64fefd-04d4-463d-bd37-832a005b9c8c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:07.662286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c1716-1227-4c90-86aa-f1aef28040cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:07.663017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c1716-1227-4c90-86aa-f1aef28040cc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.663412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c1716-1227-4c90-86aa-f1aef28040cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.663784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c1716-1227-4c90-86aa-f1aef28040cc admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.664526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c1716-1227-4c90-86aa-f1aef28040cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:07.666800 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c1716-1227-4c90-86aa-f1aef28040cc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:07.707842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 956/1910] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:07 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:07.715844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01376cb-3b57-46a6-84ec-b698c57594e9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:07.731405 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01376cb-3b57-46a6-84ec-b698c57594e9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:07.738313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcabb7-179f-447a-a5d8-59b83cfd46b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:07.739882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcabb7-179f-447a-a5d8-59b83cfd46b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.740339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcabb7-179f-447a-a5d8-59b83cfd46b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.740897 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcabb7-179f-447a-a5d8-59b83cfd46b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.742096 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcabb7-179f-447a-a5d8-59b83cfd46b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:07.744299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcabb7-179f-447a-a5d8-59b83cfd46b8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:07.764134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 955/1911] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:07.765404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c1716-1227-4c90-86aa-f1aef28040cc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:07.777430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c1716-1227-4c90-86aa-f1aef28040cc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:07.782634 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828140b-74d5-42f1-b6d7-257365958400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:07.783420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828140b-74d5-42f1-b6d7-257365958400 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.794116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828140b-74d5-42f1-b6d7-257365958400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.795834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828140b-74d5-42f1-b6d7-257365958400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.796462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5828140b-74d5-42f1-b6d7-257365958400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:07.798102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5828140b-74d5-42f1-b6d7-257365958400 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:07.825421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 957/1912] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:07.829817 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcabb7-179f-447a-a5d8-59b83cfd46b8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:07.845166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcabb7-179f-447a-a5d8-59b83cfd46b8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:07.860571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3758dfd-9c99-4710-853d-d6716bf43a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:07.861266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3758dfd-9c99-4710-853d-d6716bf43a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.861553 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3758dfd-9c99-4710-853d-d6716bf43a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.861832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3758dfd-9c99-4710-853d-d6716bf43a2e admin admin] PATH_INFO: `/v3/projects/0c8c8f2aa5f14f379ecd0938c9036ae4/users/3ebb1862c95f45b789e7c21e32547377/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.881150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3758dfd-9c99-4710-853d-d6716bf43a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c8c8f2aa5f14f379ecd0938c9036ae4, user_id=3ebb1862c95f45b789e7c21e32547377, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:07.883865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3758dfd-9c99-4710-853d-d6716bf43a2e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:07.903448 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 956/1913] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:07 2022] PUT /identity/v3/projects/0c8c8f2aa5f14f379ecd0938c9036ae4/users/3ebb1862c95f45b789e7c21e32547377/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:07.915754 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02f33b-1288-4aa1-bbe7-247a3a4befc0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:07.916054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02f33b-1288-4aa1-bbe7-247a3a4befc0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:07.916341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02f33b-1288-4aa1-bbe7-247a3a4befc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:07.980054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5e02f33b-1288-4aa1-bbe7-247a3a4befc0 None None] MFA Rules not processed for user `3ebb1862c95f45b789e7c21e32547377`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:07.981603 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5e02f33b-1288-4aa1-bbe7-247a3a4befc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:08.056518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:08.067611 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3758dfd-9c99-4710-853d-d6716bf43a2e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:08.070354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 958/1914] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:08.138481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3758dfd-9c99-4710-853d-d6716bf43a2e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:08.143410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a9426-b552-419c-9cc7-a97699d33173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:08.144030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a9426-b552-419c-9cc7-a97699d33173 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:08.144314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a9426-b552-419c-9cc7-a97699d33173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:08.145227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a9426-b552-419c-9cc7-a97699d33173 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:08.146019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a9426-b552-419c-9cc7-a97699d33173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:08.148331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a9426-b552-419c-9cc7-a97699d33173 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:08.175384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 957/1915] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:08 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:08.179945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02f33b-1288-4aa1-bbe7-247a3a4befc0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:08.190955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02f33b-1288-4aa1-bbe7-247a3a4befc0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:08.264873 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5b9a2-cd64-4c18-9451-ab102d1b4ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:08.265506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5b9a2-cd64-4c18-9451-ab102d1b4ead admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:08.265792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5b9a2-cd64-4c18-9451-ab102d1b4ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:08.266071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5b9a2-cd64-4c18-9451-ab102d1b4ead admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:08.266755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5b9a2-cd64-4c18-9451-ab102d1b4ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:08.270266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5b9a2-cd64-4c18-9451-ab102d1b4ead admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:08.307215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 959/1916] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:08 2022] POST /identity/v3/users => generated 298 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:38:08.310999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a9426-b552-419c-9cc7-a97699d33173 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:08.325634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a9426-b552-419c-9cc7-a97699d33173 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:08.331305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab691-ed45-4fc7-8248-26c8a345a377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:08.333057 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab691-ed45-4fc7-8248-26c8a345a377 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:08.333354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab691-ed45-4fc7-8248-26c8a345a377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:08.333639 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab691-ed45-4fc7-8248-26c8a345a377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:08.334354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ab691-ed45-4fc7-8248-26c8a345a377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:08.336041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ab691-ed45-4fc7-8248-26c8a345a377 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:08.349451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 958/1917] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:08.353798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5b9a2-cd64-4c18-9451-ab102d1b4ead admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:08.373220 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5b9a2-cd64-4c18-9451-ab102d1b4ead admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:08.386295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2ac7b-1754-4c42-ae73-1924eb2940dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:08.387410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2ac7b-1754-4c42-ae73-1924eb2940dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:08.387913 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2ac7b-1754-4c42-ae73-1924eb2940dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:08.388260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2ac7b-1754-4c42-ae73-1924eb2940dd admin admin] PATH_INFO: `/v3/projects/a3542d8bbb8e48da9885c713771ac5a5/users/34adafb869214cf9ae145484ee06f6f4/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:08.405186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2ac7b-1754-4c42-ae73-1924eb2940dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3542d8bbb8e48da9885c713771ac5a5, user_id=34adafb869214cf9ae145484ee06f6f4, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:08.408398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2ac7b-1754-4c42-ae73-1924eb2940dd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:08.429277 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 960/1918] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:08 2022] PUT /identity/v3/projects/a3542d8bbb8e48da9885c713771ac5a5/users/34adafb869214cf9ae145484ee06f6f4/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:08.432717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab691-ed45-4fc7-8248-26c8a345a377 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:08.447609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab691-ed45-4fc7-8248-26c8a345a377 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:08.510993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a24e34-67f5-4840-9655-a2ad1f1d49a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:08.511762 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a24e34-67f5-4840-9655-a2ad1f1d49a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:08.512149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a24e34-67f5-4840-9655-a2ad1f1d49a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:08.512529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a24e34-67f5-4840-9655-a2ad1f1d49a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:08.513332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a24e34-67f5-4840-9655-a2ad1f1d49a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:08.515042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a24e34-67f5-4840-9655-a2ad1f1d49a1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:08.526396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 959/1919] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:08.530837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2ac7b-1754-4c42-ae73-1924eb2940dd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:08.541628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2ac7b-1754-4c42-ae73-1924eb2940dd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:08.546710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759fc13-422b-4c31-84e2-510dfa9dbf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:08.547410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759fc13-422b-4c31-84e2-510dfa9dbf1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:08.547700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759fc13-422b-4c31-84e2-510dfa9dbf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:08.547985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759fc13-422b-4c31-84e2-510dfa9dbf1a admin admin] PATH_INFO: `/v3/projects/a3542d8bbb8e48da9885c713771ac5a5/users/34adafb869214cf9ae145484ee06f6f4/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:08.551349 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759fc13-422b-4c31-84e2-510dfa9dbf1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3542d8bbb8e48da9885c713771ac5a5, user_id=34adafb869214cf9ae145484ee06f6f4, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:08.554580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759fc13-422b-4c31-84e2-510dfa9dbf1a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:08.565311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 961/1920] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:08 2022] PUT /identity/v3/projects/a3542d8bbb8e48da9885c713771ac5a5/users/34adafb869214cf9ae145484ee06f6f4/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:08.577475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0595-030a-45da-b290-e4931ef9afe3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:08.577912 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0595-030a-45da-b290-e4931ef9afe3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:08.578271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0595-030a-45da-b290-e4931ef9afe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:08.587934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:08.618819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-596c0595-030a-45da-b290-e4931ef9afe3 None None] MFA Rules not processed for user `34adafb869214cf9ae145484ee06f6f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:08.620695 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-596c0595-030a-45da-b290-e4931ef9afe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:08.738698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 960/1921] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:08.785827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759fc13-422b-4c31-84e2-510dfa9dbf1a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:08.847469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759fc13-422b-4c31-84e2-510dfa9dbf1a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:08.851239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa076d4-1858-43c0-ab94-e7e33630a7f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:08.851900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa076d4-1858-43c0-ab94-e7e33630a7f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:08.852192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa076d4-1858-43c0-ab94-e7e33630a7f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:08.852477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa076d4-1858-43c0-ab94-e7e33630a7f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:08.913345 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa076d4-1858-43c0-ab94-e7e33630a7f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:08.914822 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa076d4-1858-43c0-ab94-e7e33630a7f0 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:08.929642 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 962/1922] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:15.371635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0595-030a-45da-b290-e4931ef9afe3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:15.382936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0595-030a-45da-b290-e4931ef9afe3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:15.388361 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2603d-8624-4266-9649-248d964e8861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:15.389048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2603d-8624-4266-9649-248d964e8861 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:15.389394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2603d-8624-4266-9649-248d964e8861 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:15.389747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2603d-8624-4266-9649-248d964e8861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:15.411374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2603d-8624-4266-9649-248d964e8861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:15.413807 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2603d-8624-4266-9649-248d964e8861 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:15.426606 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 961/1923] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:16.083090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa076d4-1858-43c0-ab94-e7e33630a7f0 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:16.084533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-aaa076d4-1858-43c0-ab94-e7e33630a7f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:16.091154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:16.151293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa076d4-1858-43c0-ab94-e7e33630a7f0 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:16.155942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:16.156578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:16.156862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:16.157137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] PATH_INFO: `/v3/users/f9970b66ff9d40f7a0753831a2d02ec7` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:16.158286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9970b66ff9d40f7a0753831a2d02ec7)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:16.160766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:16.210817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9970b66ff9d40f7a0753831a2d02ec7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:16.217962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9970b66ff9d40f7a0753831a2d02ec7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:16.230362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9970b66ff9d40f7a0753831a2d02ec7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:16.231006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9970b66ff9d40f7a0753831a2d02ec7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:16.231639 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9970b66ff9d40f7a0753831a2d02ec7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:16.244222 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 963/1924] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:16 2022] DELETE /identity/v3/users/f9970b66ff9d40f7a0753831a2d02ec7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:16.248782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60108d32-ba24-4a44-8444-a618f95c557a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:16.249075 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60108d32-ba24-4a44-8444-a618f95c557a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:16.249361 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60108d32-ba24-4a44-8444-a618f95c557a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:16.267930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-60108d32-ba24-4a44-8444-a618f95c557a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:16.269648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-60108d32-ba24-4a44-8444-a618f95c557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:16.326822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 962/1925] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:16.337181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:16.338587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:16.395333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd65537-91d4-448c-aef6-d081b02f6aa5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:16.398845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f0285-e44d-4c3f-a21c-ccc10cacff7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:16.399476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f0285-e44d-4c3f-a21c-ccc10cacff7e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:16.399757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f0285-e44d-4c3f-a21c-ccc10cacff7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:16.400033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f0285-e44d-4c3f-a21c-ccc10cacff7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:16.408850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f0285-e44d-4c3f-a21c-ccc10cacff7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:16.410887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f0285-e44d-4c3f-a21c-ccc10cacff7e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:16.420417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 964/1926] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:16.704568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60108d32-ba24-4a44-8444-a618f95c557a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:16.706213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-60108d32-ba24-4a44-8444-a618f95c557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:16.722484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60108d32-ba24-4a44-8444-a618f95c557a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:16.728876 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec992c8-699b-4377-9080-a4a554fca2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:16.729667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec992c8-699b-4377-9080-a4a554fca2eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:16.729954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec992c8-699b-4377-9080-a4a554fca2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:16.730231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec992c8-699b-4377-9080-a4a554fca2eb admin admin] PATH_INFO: `/v3/projects/a17ba96312b64eee99ab3d1e65d53807` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:16.731429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec992c8-699b-4377-9080-a4a554fca2eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a17ba96312b64eee99ab3d1e65d53807)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:16.733542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec992c8-699b-4377-9080-a4a554fca2eb admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:16.789073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7ec992c8-699b-4377-9080-a4a554fca2eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a17ba96312b64eee99ab3d1e65d53807', 'request_id': 'req-7ec992c8-699b-4377-9080-a4a554fca2eb'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:16.795205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7ec992c8-699b-4377-9080-a4a554fca2eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a17ba96312b64eee99ab3d1e65d53807', 'request_id': 'req-7ec992c8-699b-4377-9080-a4a554fca2eb'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:16.803217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7ec992c8-699b-4377-9080-a4a554fca2eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a17ba96312b64eee99ab3d1e65d53807', 'request_id': 'req-7ec992c8-699b-4377-9080-a4a554fca2eb'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:16.812025 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:16.819040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7ec992c8-699b-4377-9080-a4a554fca2eb admin admin] The token cache is being invalidate because project a17ba96312b64eee99ab3d1e65d53807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:16.819563 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7ec992c8-699b-4377-9080-a4a554fca2eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:16.821099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7ec992c8-699b-4377-9080-a4a554fca2eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:16.822783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 963/1927] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:16 2022] DELETE /identity/v3/projects/a17ba96312b64eee99ab3d1e65d53807 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:16.827703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0817fe1-cfdb-4d78-92b7-8d6e32cabaca None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:16.827996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0817fe1-cfdb-4d78-92b7-8d6e32cabaca None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:16.828277 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0817fe1-cfdb-4d78-92b7-8d6e32cabaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:16.850194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b0817fe1-cfdb-4d78-92b7-8d6e32cabaca None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:16.851380 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b0817fe1-cfdb-4d78-92b7-8d6e32cabaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:16.875714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:16.927814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 965/1928] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:16.936636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7b362-cac1-430e-9eb0-cace140c65ab None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:16.936925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7b362-cac1-430e-9eb0-cace140c65ab None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:16.937208 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7b362-cac1-430e-9eb0-cace140c65ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:16.958656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d4e7b362-cac1-430e-9eb0-cace140c65ab None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:16.959665 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d4e7b362-cac1-430e-9eb0-cace140c65ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:16.968188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 964/1929] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:16.973000 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0817fe1-cfdb-4d78-92b7-8d6e32cabaca None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:16.983801 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0817fe1-cfdb-4d78-92b7-8d6e32cabaca None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:16.989733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1a20d-618c-4fed-bfa7-c91104ef8c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:16.990362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1a20d-618c-4fed-bfa7-c91104ef8c72 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:16.990857 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1a20d-618c-4fed-bfa7-c91104ef8c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:16.991435 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1a20d-618c-4fed-bfa7-c91104ef8c72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:16.992276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1a20d-618c-4fed-bfa7-c91104ef8c72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:16.992689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1a20d-618c-4fed-bfa7-c91104ef8c72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:16.994820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1a20d-618c-4fed-bfa7-c91104ef8c72 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:17.008778 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 966/1930] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:38:17.014063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7b362-cac1-430e-9eb0-cace140c65ab None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:17.028322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7b362-cac1-430e-9eb0-cace140c65ab None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:17.033063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd57b8-3a6c-42ff-8a19-cc3d77704ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:17.033699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd57b8-3a6c-42ff-8a19-cc3d77704ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.034016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd57b8-3a6c-42ff-8a19-cc3d77704ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.034481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd57b8-3a6c-42ff-8a19-cc3d77704ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.035310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd57b8-3a6c-42ff-8a19-cc3d77704ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:17.037227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd57b8-3a6c-42ff-8a19-cc3d77704ae5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:17.068305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 965/1931] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:17 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:17.071843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1a20d-618c-4fed-bfa7-c91104ef8c72 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:17.083221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1a20d-618c-4fed-bfa7-c91104ef8c72 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:17.154971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cdc31-f5df-486d-a65a-4bbc0258e784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:17.155627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cdc31-f5df-486d-a65a-4bbc0258e784 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.155916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cdc31-f5df-486d-a65a-4bbc0258e784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.156344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cdc31-f5df-486d-a65a-4bbc0258e784 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.157031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cdc31-f5df-486d-a65a-4bbc0258e784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:17.158985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cdc31-f5df-486d-a65a-4bbc0258e784 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:17.183327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 967/1932] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:17 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:17.187021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd57b8-3a6c-42ff-8a19-cc3d77704ae5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:17.204115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd57b8-3a6c-42ff-8a19-cc3d77704ae5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:17.211996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848e499-3492-4c59-bc2c-d0129f58b486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:17.213004 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848e499-3492-4c59-bc2c-d0129f58b486 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.213474 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848e499-3492-4c59-bc2c-d0129f58b486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.213885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848e499-3492-4c59-bc2c-d0129f58b486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.214666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848e499-3492-4c59-bc2c-d0129f58b486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:17.217672 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848e499-3492-4c59-bc2c-d0129f58b486 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:17.235723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 966/1933] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:17.240228 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cdc31-f5df-486d-a65a-4bbc0258e784 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:17.250299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cdc31-f5df-486d-a65a-4bbc0258e784 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:17.256470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475fe3b-0313-4b99-a663-e2aed1f6c649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:17.257100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475fe3b-0313-4b99-a663-e2aed1f6c649 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.257827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475fe3b-0313-4b99-a663-e2aed1f6c649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.257947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475fe3b-0313-4b99-a663-e2aed1f6c649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.259038 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475fe3b-0313-4b99-a663-e2aed1f6c649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:17.260680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475fe3b-0313-4b99-a663-e2aed1f6c649 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:17.276274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 968/1934] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:38:17.277669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848e499-3492-4c59-bc2c-d0129f58b486 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:17.289093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848e499-3492-4c59-bc2c-d0129f58b486 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:17.293806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a0930-d50e-4f12-ae80-2c005d5dc49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:17.294487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a0930-d50e-4f12-ae80-2c005d5dc49b admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.294773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a0930-d50e-4f12-ae80-2c005d5dc49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.295055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a0930-d50e-4f12-ae80-2c005d5dc49b admin admin] PATH_INFO: `/v3/projects/8de10e4469cb4755ae16207ff995c7e5/users/5b8bec1a56434e4dbe9ff100b82c35f0/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.313422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a0930-d50e-4f12-ae80-2c005d5dc49b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8de10e4469cb4755ae16207ff995c7e5, user_id=5b8bec1a56434e4dbe9ff100b82c35f0, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:17.316708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a0930-d50e-4f12-ae80-2c005d5dc49b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:17.330461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 967/1935] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:17 2022] PUT /identity/v3/projects/8de10e4469cb4755ae16207ff995c7e5/users/5b8bec1a56434e4dbe9ff100b82c35f0/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:17.342372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce340d-2cec-4789-b957-f0e5858c2a9f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.342668 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce340d-2cec-4789-b957-f0e5858c2a9f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.342956 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce340d-2cec-4789-b957-f0e5858c2a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.373157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-51ce340d-2cec-4789-b957-f0e5858c2a9f None None] MFA Rules not processed for user `5b8bec1a56434e4dbe9ff100b82c35f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:17.375197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-51ce340d-2cec-4789-b957-f0e5858c2a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:17.455732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 969/1936] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:17.461885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a0930-d50e-4f12-ae80-2c005d5dc49b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:17.534722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a0930-d50e-4f12-ae80-2c005d5dc49b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:17.540239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18bd19-0079-4f68-94c7-336222d73d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:17.540863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18bd19-0079-4f68-94c7-336222d73d34 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.541148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18bd19-0079-4f68-94c7-336222d73d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.541428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18bd19-0079-4f68-94c7-336222d73d34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.542102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18bd19-0079-4f68-94c7-336222d73d34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:17.544075 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18bd19-0079-4f68-94c7-336222d73d34 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:17.572477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 968/1937] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:17 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:17.576474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce340d-2cec-4789-b957-f0e5858c2a9f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:17.587835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce340d-2cec-4789-b957-f0e5858c2a9f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:17.606085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:17.648053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b992c03-e05e-4753-b84a-5e7ee22cff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:17.648688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b992c03-e05e-4753-b84a-5e7ee22cff59 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.648978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b992c03-e05e-4753-b84a-5e7ee22cff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.649263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b992c03-e05e-4753-b84a-5e7ee22cff59 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.649945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b992c03-e05e-4753-b84a-5e7ee22cff59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:17.651936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b992c03-e05e-4753-b84a-5e7ee22cff59 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:17.675948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 970/1938] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:17 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:17.677073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18bd19-0079-4f68-94c7-336222d73d34 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:17.687670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18bd19-0079-4f68-94c7-336222d73d34 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:17.693330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74f284-d7e4-41d1-8bc4-62ee16168b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:17.693969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74f284-d7e4-41d1-8bc4-62ee16168b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.694257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74f284-d7e4-41d1-8bc4-62ee16168b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.694538 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74f284-d7e4-41d1-8bc4-62ee16168b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.695269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74f284-d7e4-41d1-8bc4-62ee16168b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:17.696873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74f284-d7e4-41d1-8bc4-62ee16168b76 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:17.709532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 969/1939] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:17.714184 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b992c03-e05e-4753-b84a-5e7ee22cff59 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:17.725278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b992c03-e05e-4753-b84a-5e7ee22cff59 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:17.729948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a40b8-5045-45d2-a296-e497d87ce56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:17.730603 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a40b8-5045-45d2-a296-e497d87ce56f admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.730887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a40b8-5045-45d2-a296-e497d87ce56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.731195 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a40b8-5045-45d2-a296-e497d87ce56f admin admin] PATH_INFO: `/v3/projects/f64a050a94b44c819ad9062646e1044a/users/58596ac068244aa2bd67365947b0a05c/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.746473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a40b8-5045-45d2-a296-e497d87ce56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64a050a94b44c819ad9062646e1044a, user_id=58596ac068244aa2bd67365947b0a05c, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:17.749138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a40b8-5045-45d2-a296-e497d87ce56f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:17.763931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 971/1940] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:17 2022] PUT /identity/v3/projects/f64a050a94b44c819ad9062646e1044a/users/58596ac068244aa2bd67365947b0a05c/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:38:17.767728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74f284-d7e4-41d1-8bc4-62ee16168b76 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:17.778082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74f284-d7e4-41d1-8bc4-62ee16168b76 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:17.834853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf0c28-0542-4432-afea-53cf67633325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:17.835510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf0c28-0542-4432-afea-53cf67633325 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.835810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf0c28-0542-4432-afea-53cf67633325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.836082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf0c28-0542-4432-afea-53cf67633325 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.836776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf0c28-0542-4432-afea-53cf67633325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:17.838371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf0c28-0542-4432-afea-53cf67633325 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:17.848374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 970/1941] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:17.852520 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a40b8-5045-45d2-a296-e497d87ce56f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:17.862553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a40b8-5045-45d2-a296-e497d87ce56f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:17.867618 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fd19e-71b4-4964-8e90-8cfb732092c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:17.868270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fd19e-71b4-4964-8e90-8cfb732092c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.868551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fd19e-71b4-4964-8e90-8cfb732092c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.868828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fd19e-71b4-4964-8e90-8cfb732092c8 admin admin] PATH_INFO: `/v3/projects/f64a050a94b44c819ad9062646e1044a/users/58596ac068244aa2bd67365947b0a05c/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.870740 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2fd19e-71b4-4964-8e90-8cfb732092c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64a050a94b44c819ad9062646e1044a, user_id=58596ac068244aa2bd67365947b0a05c, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:17.873396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2fd19e-71b4-4964-8e90-8cfb732092c8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:17.884665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 972/1942] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:17 2022] PUT /identity/v3/projects/f64a050a94b44c819ad9062646e1044a/users/58596ac068244aa2bd67365947b0a05c/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:38:17.894683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a91b9b-1edb-4734-8845-788b6ee024a4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:17.894973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a91b9b-1edb-4734-8845-788b6ee024a4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:17.895288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a91b9b-1edb-4734-8845-788b6ee024a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:17.903706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:17.927248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-56a91b9b-1edb-4734-8845-788b6ee024a4 None None] MFA Rules not processed for user `58596ac068244aa2bd67365947b0a05c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:17.928557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-56a91b9b-1edb-4734-8845-788b6ee024a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:18.005414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 971/1943] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:18.016016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fd19e-71b4-4964-8e90-8cfb732092c8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:18.017798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9c2fd19e-71b4-4964-8e90-8cfb732092c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:18.073952 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fd19e-71b4-4964-8e90-8cfb732092c8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:18.077827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab49c5a-afcb-4fa1-9ff3-b0bf3110768c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:18.078585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab49c5a-afcb-4fa1-9ff3-b0bf3110768c service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:18.078734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab49c5a-afcb-4fa1-9ff3-b0bf3110768c service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:18.079016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab49c5a-afcb-4fa1-9ff3-b0bf3110768c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:18.137732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab49c5a-afcb-4fa1-9ff3-b0bf3110768c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:18.139128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab49c5a-afcb-4fa1-9ff3-b0bf3110768c service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:18.147997 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 973/1944] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:20.762382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a91b9b-1edb-4734-8845-788b6ee024a4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:20.843122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a91b9b-1edb-4734-8845-788b6ee024a4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:20.849835 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:20.850812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:20.851383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:20.851918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] PATH_INFO: `/v3/users/5b8bec1a56434e4dbe9ff100b82c35f0` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:20.853992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b8bec1a56434e4dbe9ff100b82c35f0)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:20.857745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:20.922591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b8bec1a56434e4dbe9ff100b82c35f0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:20.938963 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b8bec1a56434e4dbe9ff100b82c35f0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:20.939742 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b8bec1a56434e4dbe9ff100b82c35f0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:20.948666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b8bec1a56434e4dbe9ff100b82c35f0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:20.957524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b8bec1a56434e4dbe9ff100b82c35f0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:20.973142 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 972/1945] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:20 2022] DELETE /identity/v3/users/5b8bec1a56434e4dbe9ff100b82c35f0 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:20.979380 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd00dc1-13a3-4e93-9b54-9364b234d225 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:20.979879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd00dc1-13a3-4e93-9b54-9364b234d225 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:20.979971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd00dc1-13a3-4e93-9b54-9364b234d225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:21.002799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-abd00dc1-13a3-4e93-9b54-9364b234d225 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:21.002799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-abd00dc1-13a3-4e93-9b54-9364b234d225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:21.035009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:21.075282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 974/1946] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:21.083610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:21.085048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:21.097599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:21.151883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4902881-d2c4-410b-a037-c74a60123b29 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:21.157588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82e7ef-d340-4b8b-bbf1-2a935fda9836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:21.158310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82e7ef-d340-4b8b-bbf1-2a935fda9836 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:21.158671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82e7ef-d340-4b8b-bbf1-2a935fda9836 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:21.158961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82e7ef-d340-4b8b-bbf1-2a935fda9836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:21.171108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82e7ef-d340-4b8b-bbf1-2a935fda9836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:21.173108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82e7ef-d340-4b8b-bbf1-2a935fda9836 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:21.184489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 973/1947] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:21.616566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd00dc1-13a3-4e93-9b54-9364b234d225 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:21.618070 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-abd00dc1-13a3-4e93-9b54-9364b234d225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:21.632808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd00dc1-13a3-4e93-9b54-9364b234d225 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:21.637741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:21.638918 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:21.639239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:21.639658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] PATH_INFO: `/v3/projects/8de10e4469cb4755ae16207ff995c7e5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:21.640859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8de10e4469cb4755ae16207ff995c7e5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:21.642843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:21.689269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8de10e4469cb4755ae16207ff995c7e5', 'request_id': 'req-465e073d-6c6c-4caa-a8d4-9c52670a3266'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:21.697131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8de10e4469cb4755ae16207ff995c7e5', 'request_id': 'req-465e073d-6c6c-4caa-a8d4-9c52670a3266'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:21.702900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8de10e4469cb4755ae16207ff995c7e5', 'request_id': 'req-465e073d-6c6c-4caa-a8d4-9c52670a3266'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:21.715394 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] The token cache is being invalidate because project 8de10e4469cb4755ae16207ff995c7e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:21.715756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:21.716484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:21.718321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 975/1948] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:21 2022] DELETE /identity/v3/projects/8de10e4469cb4755ae16207ff995c7e5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:38:21.722629 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82e7ef-d340-4b8b-bbf1-2a935fda9836 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:21.732246 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82e7ef-d340-4b8b-bbf1-2a935fda9836 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:21.791135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ee47d-b9b1-4c4f-aea6-fcd7f43138a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:21.791811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ee47d-b9b1-4c4f-aea6-fcd7f43138a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:21.791967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ee47d-b9b1-4c4f-aea6-fcd7f43138a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:21.792243 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ee47d-b9b1-4c4f-aea6-fcd7f43138a2 admin admin] PATH_INFO: `/v3/users/58596ac068244aa2bd67365947b0a05c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:21.793423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ee47d-b9b1-4c4f-aea6-fcd7f43138a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58596ac068244aa2bd67365947b0a05c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:21.795514 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ee47d-b9b1-4c4f-aea6-fcd7f43138a2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:21.846696 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-516ee47d-b9b1-4c4f-aea6-fcd7f43138a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58596ac068244aa2bd67365947b0a05c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:21.859804 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-516ee47d-b9b1-4c4f-aea6-fcd7f43138a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58596ac068244aa2bd67365947b0a05c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:21.860523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-516ee47d-b9b1-4c4f-aea6-fcd7f43138a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58596ac068244aa2bd67365947b0a05c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:21.868427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-516ee47d-b9b1-4c4f-aea6-fcd7f43138a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58596ac068244aa2bd67365947b0a05c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:21.875185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-516ee47d-b9b1-4c4f-aea6-fcd7f43138a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58596ac068244aa2bd67365947b0a05c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:21.881705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 974/1949] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:21 2022] DELETE /identity/v3/users/58596ac068244aa2bd67365947b0a05c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:22.163855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:22.165426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:22.181461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e073d-6c6c-4caa-a8d4-9c52670a3266 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:22.242149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ace48-8703-4d6c-ae85-71a1d39340be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:22.242797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ace48-8703-4d6c-ae85-71a1d39340be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:22.243107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ace48-8703-4d6c-ae85-71a1d39340be admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:22.243394 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ace48-8703-4d6c-ae85-71a1d39340be admin admin] PATH_INFO: `/v3/projects/f64a050a94b44c819ad9062646e1044a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:22.244558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ace48-8703-4d6c-ae85-71a1d39340be admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64a050a94b44c819ad9062646e1044a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:22.246462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ace48-8703-4d6c-ae85-71a1d39340be admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:22.292648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fd1ace48-8703-4d6c-ae85-71a1d39340be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64a050a94b44c819ad9062646e1044a', 'request_id': 'req-fd1ace48-8703-4d6c-ae85-71a1d39340be'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:22.296286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:22.300508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fd1ace48-8703-4d6c-ae85-71a1d39340be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64a050a94b44c819ad9062646e1044a', 'request_id': 'req-fd1ace48-8703-4d6c-ae85-71a1d39340be'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:22.307713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fd1ace48-8703-4d6c-ae85-71a1d39340be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64a050a94b44c819ad9062646e1044a', 'request_id': 'req-fd1ace48-8703-4d6c-ae85-71a1d39340be'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:22.319679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fd1ace48-8703-4d6c-ae85-71a1d39340be admin admin] The token cache is being invalidate because project f64a050a94b44c819ad9062646e1044a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:22.319969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fd1ace48-8703-4d6c-ae85-71a1d39340be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:22.320611 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fd1ace48-8703-4d6c-ae85-71a1d39340be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:22.322111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 976/1950] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:22 2022] DELETE /identity/v3/projects/f64a050a94b44c819ad9062646e1044a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:22.326903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d52e48-1dc7-4606-b9b7-0b247e7cfd7c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:22.327232 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d52e48-1dc7-4606-b9b7-0b247e7cfd7c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:22.327549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d52e48-1dc7-4606-b9b7-0b247e7cfd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:22.348683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-37d52e48-1dc7-4606-b9b7-0b247e7cfd7c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:22.349911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-37d52e48-1dc7-4606-b9b7-0b247e7cfd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:22.375957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:22.416813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 975/1951] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:22.425753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259b48c-54a9-41e3-8048-73dde81f04a7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:22.426048 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259b48c-54a9-41e3-8048-73dde81f04a7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:22.426332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259b48c-54a9-41e3-8048-73dde81f04a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:22.449941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3259b48c-54a9-41e3-8048-73dde81f04a7 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:22.451203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3259b48c-54a9-41e3-8048-73dde81f04a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:22.461628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 977/1952] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:22.465593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d52e48-1dc7-4606-b9b7-0b247e7cfd7c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:22.476790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d52e48-1dc7-4606-b9b7-0b247e7cfd7c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:22.481827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff94829-531a-4fc3-95e6-1d78ac2d0dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:22.482464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff94829-531a-4fc3-95e6-1d78ac2d0dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:22.482937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff94829-531a-4fc3-95e6-1d78ac2d0dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:22.483247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff94829-531a-4fc3-95e6-1d78ac2d0dc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:22.483954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff94829-531a-4fc3-95e6-1d78ac2d0dc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:22.485578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff94829-531a-4fc3-95e6-1d78ac2d0dc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:22.487240 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff94829-531a-4fc3-95e6-1d78ac2d0dc1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:22.506656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 976/1953] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:38:22.508152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259b48c-54a9-41e3-8048-73dde81f04a7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:22.520183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259b48c-54a9-41e3-8048-73dde81f04a7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:22.525496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb568ff7-5e2d-4c31-949a-ef5ca58e9a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:22.526124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb568ff7-5e2d-4c31-949a-ef5ca58e9a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:22.526412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb568ff7-5e2d-4c31-949a-ef5ca58e9a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:22.526696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb568ff7-5e2d-4c31-949a-ef5ca58e9a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:22.527417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb568ff7-5e2d-4c31-949a-ef5ca58e9a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:22.529446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb568ff7-5e2d-4c31-949a-ef5ca58e9a75 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:22.572474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 978/1954] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:22 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:22.573203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff94829-531a-4fc3-95e6-1d78ac2d0dc1 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:22.589436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff94829-531a-4fc3-95e6-1d78ac2d0dc1 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:22.678413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265f0a3-ca4e-431f-b457-7ca16d89084c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:22.679032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265f0a3-ca4e-431f-b457-7ca16d89084c admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:22.679346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265f0a3-ca4e-431f-b457-7ca16d89084c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:22.679624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265f0a3-ca4e-431f-b457-7ca16d89084c admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:22.680379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265f0a3-ca4e-431f-b457-7ca16d89084c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:22.682381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265f0a3-ca4e-431f-b457-7ca16d89084c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:22.717113 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 977/1955] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:22 2022] POST /identity/v3/users => generated 286 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:38:22.724294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb568ff7-5e2d-4c31-949a-ef5ca58e9a75 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:22.734858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb568ff7-5e2d-4c31-949a-ef5ca58e9a75 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:22.739784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf628d-2e7c-4a73-a28b-ebf2ba21ac41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:22.740428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf628d-2e7c-4a73-a28b-ebf2ba21ac41 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:22.740733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf628d-2e7c-4a73-a28b-ebf2ba21ac41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:22.741022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf628d-2e7c-4a73-a28b-ebf2ba21ac41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:22.741784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf628d-2e7c-4a73-a28b-ebf2ba21ac41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:22.743533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf628d-2e7c-4a73-a28b-ebf2ba21ac41 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:22.754678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 979/1956] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:22.758870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265f0a3-ca4e-431f-b457-7ca16d89084c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:22.768811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265f0a3-ca4e-431f-b457-7ca16d89084c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:22.774146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83692d2b-95a8-4aea-9e5d-928cb9691cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:22.774756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83692d2b-95a8-4aea-9e5d-928cb9691cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:22.775136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83692d2b-95a8-4aea-9e5d-928cb9691cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:22.775507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83692d2b-95a8-4aea-9e5d-928cb9691cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:22.776210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83692d2b-95a8-4aea-9e5d-928cb9691cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:22.777874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83692d2b-95a8-4aea-9e5d-928cb9691cf0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:22.788689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 978/1957] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:22.792744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf628d-2e7c-4a73-a28b-ebf2ba21ac41 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:22.802821 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf628d-2e7c-4a73-a28b-ebf2ba21ac41 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:22.807373 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c17817-273d-4466-85c9-23cba2a6e513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:22.808034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c17817-273d-4466-85c9-23cba2a6e513 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:22.808318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c17817-273d-4466-85c9-23cba2a6e513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:22.808603 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c17817-273d-4466-85c9-23cba2a6e513 admin admin] PATH_INFO: `/v3/projects/a5b0cb87dfc4461e9911c7e62b6ab959/users/9255edea8b504727a587443e124f3e7a/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:22.823335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c17817-273d-4466-85c9-23cba2a6e513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0cb87dfc4461e9911c7e62b6ab959, user_id=9255edea8b504727a587443e124f3e7a, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:22.825995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c17817-273d-4466-85c9-23cba2a6e513 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:22.839588 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 980/1958] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:22 2022] PUT /identity/v3/projects/a5b0cb87dfc4461e9911c7e62b6ab959/users/9255edea8b504727a587443e124f3e7a/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:22.844342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019e1be-b003-4da4-8a50-e433521e85a1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:22.844805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019e1be-b003-4da4-8a50-e433521e85a1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:22.844927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019e1be-b003-4da4-8a50-e433521e85a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:22.863857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5019e1be-b003-4da4-8a50-e433521e85a1 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:22.869113 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5019e1be-b003-4da4-8a50-e433521e85a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:22.927883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 979/1959] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:22.937544 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c17817-273d-4466-85c9-23cba2a6e513 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:22.939347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-76c17817-273d-4466-85c9-23cba2a6e513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:22.997360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c17817-273d-4466-85c9-23cba2a6e513 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:23.001015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e5e90-655a-4ee9-9c6e-d159c2723496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:23.001631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e5e90-655a-4ee9-9c6e-d159c2723496 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:23.001917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e5e90-655a-4ee9-9c6e-d159c2723496 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:23.002196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e5e90-655a-4ee9-9c6e-d159c2723496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:23.012116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e5e90-655a-4ee9-9c6e-d159c2723496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:23.013539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e5e90-655a-4ee9-9c6e-d159c2723496 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:23.022981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 981/1960] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:23.762798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59835c41-075b-4da1-a6ad-42163bc6346f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:23.763259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59835c41-075b-4da1-a6ad-42163bc6346f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:23.763583 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59835c41-075b-4da1-a6ad-42163bc6346f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:23.792213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-59835c41-075b-4da1-a6ad-42163bc6346f None None] MFA Rules not processed for user `9255edea8b504727a587443e124f3e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:23.793427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-59835c41-075b-4da1-a6ad-42163bc6346f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:23.820732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:23.861559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 980/1961] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:23.873012 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e5e90-655a-4ee9-9c6e-d159c2723496 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:23.874416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f95e5e90-655a-4ee9-9c6e-d159c2723496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:23.931921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e5e90-655a-4ee9-9c6e-d159c2723496 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:23.935570 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5510219-3adc-4a10-a607-3bb29e806186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:23.936312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5510219-3adc-4a10-a607-3bb29e806186 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:23.936598 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5510219-3adc-4a10-a607-3bb29e806186 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:23.936875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5510219-3adc-4a10-a607-3bb29e806186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:23.946657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5510219-3adc-4a10-a607-3bb29e806186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:23.948284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5510219-3adc-4a10-a607-3bb29e806186 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:23.958142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 982/1962] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:26.357916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf85d30-e462-42b6-8aba-3cbb8d09b7a3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:26.358460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf85d30-e462-42b6-8aba-3cbb8d09b7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:26.358569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf85d30-e462-42b6-8aba-3cbb8d09b7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:26.377643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8bf85d30-e462-42b6-8aba-3cbb8d09b7a3 None None] MFA Rules not processed for user `9255edea8b504727a587443e124f3e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:26.379420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8bf85d30-e462-42b6-8aba-3cbb8d09b7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:26.386601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 981/1963] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:26.915799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5510219-3adc-4a10-a607-3bb29e806186 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:26.921842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a5510219-3adc-4a10-a607-3bb29e806186 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:26.984429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5510219-3adc-4a10-a607-3bb29e806186 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:26.993444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:26.994055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:26.994349 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:26.994634 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] PATH_INFO: `/v3/users/3ebb1862c95f45b789e7c21e32547377` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:26.995896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ebb1862c95f45b789e7c21e32547377)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:26.998053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:27.026407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:27.072362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ebb1862c95f45b789e7c21e32547377'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.083323 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ebb1862c95f45b789e7c21e32547377'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.098211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ebb1862c95f45b789e7c21e32547377'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.098953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ebb1862c95f45b789e7c21e32547377'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.099609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ebb1862c95f45b789e7c21e32547377'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.116669 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 983/1964] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:26 2022] DELETE /identity/v3/users/3ebb1862c95f45b789e7c21e32547377 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:27.121238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26d441-d61d-42b5-9494-9bbce2ae94ec None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:27.121599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26d441-d61d-42b5-9494-9bbce2ae94ec None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:27.122028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26d441-d61d-42b5-9494-9bbce2ae94ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:27.146596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7d26d441-d61d-42b5-9494-9bbce2ae94ec None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:27.147951 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7d26d441-d61d-42b5-9494-9bbce2ae94ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:27.277359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 982/1965] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:27.279927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:27.281564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:27.368654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481ba1f-9bfa-4cf2-a157-fadb11657572 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:27.372202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dafee8-4526-4d0f-8733-94908a0e6f9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:27.372816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dafee8-4526-4d0f-8733-94908a0e6f9a service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:27.373099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dafee8-4526-4d0f-8733-94908a0e6f9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:27.373375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dafee8-4526-4d0f-8733-94908a0e6f9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:27.382888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dafee8-4526-4d0f-8733-94908a0e6f9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:27.384420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dafee8-4526-4d0f-8733-94908a0e6f9a service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:27.394137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 984/1966] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:27.669122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26d441-d61d-42b5-9494-9bbce2ae94ec None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:27.670574 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7d26d441-d61d-42b5-9494-9bbce2ae94ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:27.682914 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26d441-d61d-42b5-9494-9bbce2ae94ec None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:27.687791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:27.688450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:27.688743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:27.689031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] PATH_INFO: `/v3/projects/0c8c8f2aa5f14f379ecd0938c9036ae4` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:27.690629 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c8c8f2aa5f14f379ecd0938c9036ae4)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:27.693345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:27.749452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c8c8f2aa5f14f379ecd0938c9036ae4', 'request_id': 'req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.760626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c8c8f2aa5f14f379ecd0938c9036ae4', 'request_id': 'req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.767707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c8c8f2aa5f14f379ecd0938c9036ae4', 'request_id': 'req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.785374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] The token cache is being invalidate because project 0c8c8f2aa5f14f379ecd0938c9036ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:27.785680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.787138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.790017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 983/1967] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:27 2022] DELETE /identity/v3/projects/0c8c8f2aa5f14f379ecd0938c9036ae4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:27.793611 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dafee8-4526-4d0f-8733-94908a0e6f9a service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:27.808596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dafee8-4526-4d0f-8733-94908a0e6f9a service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:27.886978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1d111-d2eb-4747-afeb-bcd6f2f534e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:27.887693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1d111-d2eb-4747-afeb-bcd6f2f534e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:27.887978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1d111-d2eb-4747-afeb-bcd6f2f534e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:27.888253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1d111-d2eb-4747-afeb-bcd6f2f534e3 admin admin] PATH_INFO: `/v3/users/34adafb869214cf9ae145484ee06f6f4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:27.890505 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1d111-d2eb-4747-afeb-bcd6f2f534e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34adafb869214cf9ae145484ee06f6f4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:27.891380 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1d111-d2eb-4747-afeb-bcd6f2f534e3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:27.935627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:27.941502 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1fa1d111-d2eb-4747-afeb-bcd6f2f534e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34adafb869214cf9ae145484ee06f6f4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.955595 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1fa1d111-d2eb-4747-afeb-bcd6f2f534e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34adafb869214cf9ae145484ee06f6f4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.969615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1fa1d111-d2eb-4747-afeb-bcd6f2f534e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34adafb869214cf9ae145484ee06f6f4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.970331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1fa1d111-d2eb-4747-afeb-bcd6f2f534e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34adafb869214cf9ae145484ee06f6f4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.970940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1fa1d111-d2eb-4747-afeb-bcd6f2f534e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34adafb869214cf9ae145484ee06f6f4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:27.987333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 985/1968] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:27 2022] DELETE /identity/v3/users/34adafb869214cf9ae145484ee06f6f4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:28.381100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:28.383314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:28.412156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffb3e3-19f5-4cee-a6d2-1fba0542cab7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:28.494049 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:28.537204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088ba6b-f6f9-48be-9da9-9d9128604af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:28.537953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ba6b-f6f9-48be-9da9-9d9128604af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:28.538306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ba6b-f6f9-48be-9da9-9d9128604af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:28.538649 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ba6b-f6f9-48be-9da9-9d9128604af8 admin admin] PATH_INFO: `/v3/projects/a3542d8bbb8e48da9885c713771ac5a5` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:28.540847 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088ba6b-f6f9-48be-9da9-9d9128604af8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3542d8bbb8e48da9885c713771ac5a5)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:28.543167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088ba6b-f6f9-48be-9da9-9d9128604af8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:28.596184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4088ba6b-f6f9-48be-9da9-9d9128604af8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3542d8bbb8e48da9885c713771ac5a5', 'request_id': 'req-4088ba6b-f6f9-48be-9da9-9d9128604af8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:28.602702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4088ba6b-f6f9-48be-9da9-9d9128604af8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3542d8bbb8e48da9885c713771ac5a5', 'request_id': 'req-4088ba6b-f6f9-48be-9da9-9d9128604af8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:28.609358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4088ba6b-f6f9-48be-9da9-9d9128604af8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3542d8bbb8e48da9885c713771ac5a5', 'request_id': 'req-4088ba6b-f6f9-48be-9da9-9d9128604af8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:28.624596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4088ba6b-f6f9-48be-9da9-9d9128604af8 admin admin] The token cache is being invalidate because project a3542d8bbb8e48da9885c713771ac5a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:28.624830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4088ba6b-f6f9-48be-9da9-9d9128604af8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:28.625459 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4088ba6b-f6f9-48be-9da9-9d9128604af8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:28.627133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 984/1969] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:28 2022] DELETE /identity/v3/projects/a3542d8bbb8e48da9885c713771ac5a5 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:38:28.634481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2b718-f4ff-4882-baf6-f88f56a630a8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:28.634827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2b718-f4ff-4882-baf6-f88f56a630a8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:28.635290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2b718-f4ff-4882-baf6-f88f56a630a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:28.656671 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-83f2b718-f4ff-4882-baf6-f88f56a630a8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:28.657996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-83f2b718-f4ff-4882-baf6-f88f56a630a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:28.729553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 986/1970] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:28.738293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92915ee3-b210-42ff-9543-f1fbc0988e11 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:28.738808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92915ee3-b210-42ff-9543-f1fbc0988e11 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:28.738984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92915ee3-b210-42ff-9543-f1fbc0988e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:28.758971 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-92915ee3-b210-42ff-9543-f1fbc0988e11 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:28.760250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-92915ee3-b210-42ff-9543-f1fbc0988e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:28.768347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 985/1971] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:28.772971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2b718-f4ff-4882-baf6-f88f56a630a8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:28.783705 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2b718-f4ff-4882-baf6-f88f56a630a8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:28.789346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac64ba1-8105-4425-a199-09d8b0e60538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:28.790054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac64ba1-8105-4425-a199-09d8b0e60538 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:28.790417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac64ba1-8105-4425-a199-09d8b0e60538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:28.790777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac64ba1-8105-4425-a199-09d8b0e60538 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:28.791597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac64ba1-8105-4425-a199-09d8b0e60538 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:28.792088 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac64ba1-8105-4425-a199-09d8b0e60538 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:28.793888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac64ba1-8105-4425-a199-09d8b0e60538 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:28.807983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 987/1972] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:38:28.812225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92915ee3-b210-42ff-9543-f1fbc0988e11 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:28.823751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92915ee3-b210-42ff-9543-f1fbc0988e11 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:28.832668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd17d0-dc5c-4c54-8d5c-3e12794f7ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:28.833690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd17d0-dc5c-4c54-8d5c-3e12794f7ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:28.834133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd17d0-dc5c-4c54-8d5c-3e12794f7ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:28.835145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd17d0-dc5c-4c54-8d5c-3e12794f7ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:28.836143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd17d0-dc5c-4c54-8d5c-3e12794f7ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:28.838925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd17d0-dc5c-4c54-8d5c-3e12794f7ab3 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:28.882378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac64ba1-8105-4425-a199-09d8b0e60538 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:28.894808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 986/1973] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:28 2022] POST /identity/v3/projects => generated 359 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:38:28.895223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac64ba1-8105-4425-a199-09d8b0e60538 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:28.974279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304144a3-8e0a-4f3b-a1d8-5682403a3cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:28.974279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304144a3-8e0a-4f3b-a1d8-5682403a3cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:28.975165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304144a3-8e0a-4f3b-a1d8-5682403a3cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:28.975165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304144a3-8e0a-4f3b-a1d8-5682403a3cbc admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:28.976245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304144a3-8e0a-4f3b-a1d8-5682403a3cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:28.978408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304144a3-8e0a-4f3b-a1d8-5682403a3cbc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:29.011126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 988/1974] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:28 2022] POST /identity/v3/users => generated 280 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:29.012741 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd17d0-dc5c-4c54-8d5c-3e12794f7ab3 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:29.032951 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd17d0-dc5c-4c54-8d5c-3e12794f7ab3 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:29.039478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c900e-8be6-4b5f-a189-23770490984c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:29.040039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c900e-8be6-4b5f-a189-23770490984c admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:29.040369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c900e-8be6-4b5f-a189-23770490984c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:29.042907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c900e-8be6-4b5f-a189-23770490984c admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:29.045113 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c900e-8be6-4b5f-a189-23770490984c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:29.051668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c900e-8be6-4b5f-a189-23770490984c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:29.074584 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 987/1975] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:29.078555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304144a3-8e0a-4f3b-a1d8-5682403a3cbc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:29.090551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304144a3-8e0a-4f3b-a1d8-5682403a3cbc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:29.095846 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24448637-00e0-46f6-8da6-2b055d4556c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:29.097063 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448637-00e0-46f6-8da6-2b055d4556c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:29.097795 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448637-00e0-46f6-8da6-2b055d4556c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:29.098354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448637-00e0-46f6-8da6-2b055d4556c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:29.099401 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24448637-00e0-46f6-8da6-2b055d4556c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:29.101332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24448637-00e0-46f6-8da6-2b055d4556c8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:29.119726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 989/1976] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:38:29.124212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c900e-8be6-4b5f-a189-23770490984c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:29.136645 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c900e-8be6-4b5f-a189-23770490984c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:29.142710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a2d67-69dc-4735-a78b-ac2aa0f92248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:29.144227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a2d67-69dc-4735-a78b-ac2aa0f92248 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:29.145196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a2d67-69dc-4735-a78b-ac2aa0f92248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:29.145635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a2d67-69dc-4735-a78b-ac2aa0f92248 admin admin] PATH_INFO: `/v3/projects/ae2b14440d9e47b28bce1c1a4ccb32b5/users/3cd781629e0e45178901dd1e75a0d5b5/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:29.179123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a2d67-69dc-4735-a78b-ac2aa0f92248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2b14440d9e47b28bce1c1a4ccb32b5, user_id=3cd781629e0e45178901dd1e75a0d5b5, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:29.182152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a2d67-69dc-4735-a78b-ac2aa0f92248 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:29.205838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 988/1977] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:29 2022] PUT /identity/v3/projects/ae2b14440d9e47b28bce1c1a4ccb32b5/users/3cd781629e0e45178901dd1e75a0d5b5/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:29.218612 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a28e14-5f37-44fd-ac95-75837bd89003 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:29.218908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a28e14-5f37-44fd-ac95-75837bd89003 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:29.219217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a28e14-5f37-44fd-ac95-75837bd89003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:29.249349 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-44a28e14-5f37-44fd-ac95-75837bd89003 None None] MFA Rules not processed for user `3cd781629e0e45178901dd1e75a0d5b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:29.250803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-44a28e14-5f37-44fd-ac95-75837bd89003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:29.266363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:29.333011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 990/1978] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:29 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:29.377101 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a2d67-69dc-4735-a78b-ac2aa0f92248 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:29.379958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1d7a2d67-69dc-4735-a78b-ac2aa0f92248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:29.462433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a2d67-69dc-4735-a78b-ac2aa0f92248 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:29.476370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8508257-2214-4e39-bb5e-0cfa018a8e32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:29.476864 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8508257-2214-4e39-bb5e-0cfa018a8e32 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:29.477148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8508257-2214-4e39-bb5e-0cfa018a8e32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:29.477566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8508257-2214-4e39-bb5e-0cfa018a8e32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:29.495674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8508257-2214-4e39-bb5e-0cfa018a8e32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:29.495674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8508257-2214-4e39-bb5e-0cfa018a8e32 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:29.537657 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 989/1979] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:29 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:29.545901 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a28e14-5f37-44fd-ac95-75837bd89003 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:29.623466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a28e14-5f37-44fd-ac95-75837bd89003 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:29.628294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:29.628946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:29.629232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:29.629529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] PATH_INFO: `/v3/users/3cd781629e0e45178901dd1e75a0d5b5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:29.630944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cd781629e0e45178901dd1e75a0d5b5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:29.632926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:29.687013 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cd781629e0e45178901dd1e75a0d5b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:29.698869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cd781629e0e45178901dd1e75a0d5b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:29.712808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cd781629e0e45178901dd1e75a0d5b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:29.713761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cd781629e0e45178901dd1e75a0d5b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:29.714650 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cd781629e0e45178901dd1e75a0d5b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:29.728466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 991/1980] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:29 2022] DELETE /identity/v3/users/3cd781629e0e45178901dd1e75a0d5b5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:29.732981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01fa6a-29d2-4f68-816b-a9fa8cae62b2 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:29.733285 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01fa6a-29d2-4f68-816b-a9fa8cae62b2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:29.733572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01fa6a-29d2-4f68-816b-a9fa8cae62b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:29.752729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8b01fa6a-29d2-4f68-816b-a9fa8cae62b2 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:29.753986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8b01fa6a-29d2-4f68-816b-a9fa8cae62b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:29.762508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:29.815470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 990/1981] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:29.825127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:29.826540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:29.863594 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:29.886979 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d4d60-9f20-4c08-b768-319c196c1a65 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:29.890704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401af31-f84a-48e4-8de4-1f81d84887e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:29.891467 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401af31-f84a-48e4-8de4-1f81d84887e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:29.891723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401af31-f84a-48e4-8de4-1f81d84887e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:29.892006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401af31-f84a-48e4-8de4-1f81d84887e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:29.901868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401af31-f84a-48e4-8de4-1f81d84887e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:29.903461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401af31-f84a-48e4-8de4-1f81d84887e9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:29.914443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 992/1982] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:30.194216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01fa6a-29d2-4f68-816b-a9fa8cae62b2 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:30.196079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8b01fa6a-29d2-4f68-816b-a9fa8cae62b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:30.219337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01fa6a-29d2-4f68-816b-a9fa8cae62b2 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:30.224954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61514766-08f0-4e32-bb52-922b9bdd32e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:30.226029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61514766-08f0-4e32-bb52-922b9bdd32e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:30.226449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61514766-08f0-4e32-bb52-922b9bdd32e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:30.226852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61514766-08f0-4e32-bb52-922b9bdd32e4 admin admin] PATH_INFO: `/v3/projects/ae2b14440d9e47b28bce1c1a4ccb32b5` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:30.228168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61514766-08f0-4e32-bb52-922b9bdd32e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae2b14440d9e47b28bce1c1a4ccb32b5)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:30.230118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61514766-08f0-4e32-bb52-922b9bdd32e4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:30.278506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61514766-08f0-4e32-bb52-922b9bdd32e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae2b14440d9e47b28bce1c1a4ccb32b5', 'request_id': 'req-61514766-08f0-4e32-bb52-922b9bdd32e4'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:30.284095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61514766-08f0-4e32-bb52-922b9bdd32e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae2b14440d9e47b28bce1c1a4ccb32b5', 'request_id': 'req-61514766-08f0-4e32-bb52-922b9bdd32e4'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:30.290233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61514766-08f0-4e32-bb52-922b9bdd32e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae2b14440d9e47b28bce1c1a4ccb32b5', 'request_id': 'req-61514766-08f0-4e32-bb52-922b9bdd32e4'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:30.303771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61514766-08f0-4e32-bb52-922b9bdd32e4 admin admin] The token cache is being invalidate because project ae2b14440d9e47b28bce1c1a4ccb32b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:30.304065 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61514766-08f0-4e32-bb52-922b9bdd32e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:30.304694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61514766-08f0-4e32-bb52-922b9bdd32e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:30.306385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 991/1983] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:30 2022] DELETE /identity/v3/projects/ae2b14440d9e47b28bce1c1a4ccb32b5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:30.313213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0d5a4-021d-4d71-800d-7fcf11db849e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:30.313213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0d5a4-021d-4d71-800d-7fcf11db849e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:30.313847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0d5a4-021d-4d71-800d-7fcf11db849e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:30.335006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2af0d5a4-021d-4d71-800d-7fcf11db849e None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:30.336340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2af0d5a4-021d-4d71-800d-7fcf11db849e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:30.408294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 993/1984] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:30.417745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027fda4-bdf7-44b5-bc25-bb5c627c744f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:30.418045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027fda4-bdf7-44b5-bc25-bb5c627c744f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:30.418334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027fda4-bdf7-44b5-bc25-bb5c627c744f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:30.438134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7027fda4-bdf7-44b5-bc25-bb5c627c744f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:30.439320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7027fda4-bdf7-44b5-bc25-bb5c627c744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:30.448139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 992/1985] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:30.452316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0d5a4-021d-4d71-800d-7fcf11db849e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:30.462824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0d5a4-021d-4d71-800d-7fcf11db849e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:30.467518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914192c7-887c-429c-9019-c0c9aebd8670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:30.468133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914192c7-887c-429c-9019-c0c9aebd8670 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:30.468425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914192c7-887c-429c-9019-c0c9aebd8670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:30.468713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914192c7-887c-429c-9019-c0c9aebd8670 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:30.469410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914192c7-887c-429c-9019-c0c9aebd8670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:30.469841 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914192c7-887c-429c-9019-c0c9aebd8670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:30.471481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914192c7-887c-429c-9019-c0c9aebd8670 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:30.483687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 994/1986] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:38:30.488709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027fda4-bdf7-44b5-bc25-bb5c627c744f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:30.498848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027fda4-bdf7-44b5-bc25-bb5c627c744f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:30.508358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debf3c8-d93c-4e67-90a0-6d5247772a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:30.509014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debf3c8-d93c-4e67-90a0-6d5247772a73 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:30.509296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debf3c8-d93c-4e67-90a0-6d5247772a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:30.509632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debf3c8-d93c-4e67-90a0-6d5247772a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:30.510268 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debf3c8-d93c-4e67-90a0-6d5247772a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:30.512195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debf3c8-d93c-4e67-90a0-6d5247772a73 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:30.549109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 993/1987] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:30 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:30.553498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914192c7-887c-429c-9019-c0c9aebd8670 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:30.565094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914192c7-887c-429c-9019-c0c9aebd8670 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:30.645212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8f8c1-8e1b-436f-b971-ed17677d7d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:30.646657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8f8c1-8e1b-436f-b971-ed17677d7d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:30.647055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8f8c1-8e1b-436f-b971-ed17677d7d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:30.647476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8f8c1-8e1b-436f-b971-ed17677d7d16 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:30.648288 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8f8c1-8e1b-436f-b971-ed17677d7d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:30.651455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8f8c1-8e1b-436f-b971-ed17677d7d16 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:30.683115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 995/1988] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:30 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:30.686760 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debf3c8-d93c-4e67-90a0-6d5247772a73 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:30.698502 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debf3c8-d93c-4e67-90a0-6d5247772a73 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:30.702993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80299cd-c88b-430e-acf6-476e5d9c227e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:30.703706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80299cd-c88b-430e-acf6-476e5d9c227e admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:30.703946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80299cd-c88b-430e-acf6-476e5d9c227e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:30.704227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80299cd-c88b-430e-acf6-476e5d9c227e admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:30.705243 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80299cd-c88b-430e-acf6-476e5d9c227e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:30.706777 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80299cd-c88b-430e-acf6-476e5d9c227e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:30.720082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 994/1989] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:30.724270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8f8c1-8e1b-436f-b971-ed17677d7d16 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:30.734224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8f8c1-8e1b-436f-b971-ed17677d7d16 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:30.740516 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf29dc5-1df0-4565-bd66-9d91f8115d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:30.740708 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf29dc5-1df0-4565-bd66-9d91f8115d0a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:30.741126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf29dc5-1df0-4565-bd66-9d91f8115d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:30.741411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf29dc5-1df0-4565-bd66-9d91f8115d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:30.742110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf29dc5-1df0-4565-bd66-9d91f8115d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:30.743986 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf29dc5-1df0-4565-bd66-9d91f8115d0a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:30.759262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 996/1990] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:30.760488 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80299cd-c88b-430e-acf6-476e5d9c227e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:30.774435 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80299cd-c88b-430e-acf6-476e5d9c227e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:30.779779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf666f6-558f-4e1e-ab67-0170e9b7f611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:30.780464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf666f6-558f-4e1e-ab67-0170e9b7f611 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:30.781087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf666f6-558f-4e1e-ab67-0170e9b7f611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:30.781087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf666f6-558f-4e1e-ab67-0170e9b7f611 admin admin] PATH_INFO: `/v3/projects/f07c6f16ad7747c5a42b4d0351a44545/users/d714badcdf8e4461ae1f7ebe4f0c82d4/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:30.803960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf666f6-558f-4e1e-ab67-0170e9b7f611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f07c6f16ad7747c5a42b4d0351a44545, user_id=d714badcdf8e4461ae1f7ebe4f0c82d4, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:30.806969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf666f6-558f-4e1e-ab67-0170e9b7f611 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:30.826633 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 995/1991] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:30 2022] PUT /identity/v3/projects/f07c6f16ad7747c5a42b4d0351a44545/users/d714badcdf8e4461ae1f7ebe4f0c82d4/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:30.831979 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee460cd-668a-4662-be77-7733b389db71 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:30.832274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee460cd-668a-4662-be77-7733b389db71 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:30.832558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee460cd-668a-4662-be77-7733b389db71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:30.856303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8ee460cd-668a-4662-be77-7733b389db71 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:30.857543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8ee460cd-668a-4662-be77-7733b389db71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:30.884590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf666f6-558f-4e1e-ab67-0170e9b7f611 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:30.887698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cbf666f6-558f-4e1e-ab67-0170e9b7f611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:30.906539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:30.928346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:30.934972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 997/1992] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:30.942989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee460cd-668a-4662-be77-7733b389db71 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:30.944465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8ee460cd-668a-4662-be77-7733b389db71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:30.983561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf666f6-558f-4e1e-ab67-0170e9b7f611 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:30.988978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:30.989635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:30.989921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:30.990206 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] PATH_INFO: `/v3/users/bdd92e8b7be74f8696b881dc20e2aabf` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:30.991955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd92e8b7be74f8696b881dc20e2aabf)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:30.994283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:31.046463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee460cd-668a-4662-be77-7733b389db71 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:31.051490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2963e-0bb4-4091-bb5b-e5ba63b8a856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:31.052545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2963e-0bb4-4091-bb5b-e5ba63b8a856 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:31.052835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2963e-0bb4-4091-bb5b-e5ba63b8a856 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:31.053127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2963e-0bb4-4091-bb5b-e5ba63b8a856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:31.064624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2963e-0bb4-4091-bb5b-e5ba63b8a856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:31.066830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2963e-0bb4-4091-bb5b-e5ba63b8a856 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:31.079121 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 998/1993] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:38:31.084743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd92e8b7be74f8696b881dc20e2aabf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:31.101133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd92e8b7be74f8696b881dc20e2aabf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:31.101906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd92e8b7be74f8696b881dc20e2aabf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:31.111217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd92e8b7be74f8696b881dc20e2aabf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:31.119953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd92e8b7be74f8696b881dc20e2aabf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:31.131216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 996/1994] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:30 2022] DELETE /identity/v3/users/bdd92e8b7be74f8696b881dc20e2aabf => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:31.132835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e146f5-adab-4a97-9259-a65cd0e6bd20 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:31.133132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e146f5-adab-4a97-9259-a65cd0e6bd20 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:31.133443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e146f5-adab-4a97-9259-a65cd0e6bd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:31.164874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a9e146f5-adab-4a97-9259-a65cd0e6bd20 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:31.166450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a9e146f5-adab-4a97-9259-a65cd0e6bd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:31.246198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 999/1995] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:31.264923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:31.266934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:31.341068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdced1c3-29ae-4c4c-9837-77e5c44cd8fd admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:31.346028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8820138-42e5-4f4e-915f-f732582a2a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:31.347432 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8820138-42e5-4f4e-915f-f732582a2a44 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:31.347782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8820138-42e5-4f4e-915f-f732582a2a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:31.348136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8820138-42e5-4f4e-915f-f732582a2a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:31.360431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8820138-42e5-4f4e-915f-f732582a2a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:31.362136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8820138-42e5-4f4e-915f-f732582a2a44 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:31.375161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 997/1996] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:32.175731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e146f5-adab-4a97-9259-a65cd0e6bd20 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:32.177500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a9e146f5-adab-4a97-9259-a65cd0e6bd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:32.193199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e146f5-adab-4a97-9259-a65cd0e6bd20 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:32.201890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfa161-b4a8-44c6-9fcb-05eb661413f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:32.202792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfa161-b4a8-44c6-9fcb-05eb661413f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:32.203276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfa161-b4a8-44c6-9fcb-05eb661413f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:32.203748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfa161-b4a8-44c6-9fcb-05eb661413f8 admin admin] PATH_INFO: `/v3/projects/63fb4664783f4667ba10d567e5c0c5be` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:32.205449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfa161-b4a8-44c6-9fcb-05eb661413f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63fb4664783f4667ba10d567e5c0c5be)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:32.207512 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfa161-b4a8-44c6-9fcb-05eb661413f8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:32.272281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-badfa161-b4a8-44c6-9fcb-05eb661413f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63fb4664783f4667ba10d567e5c0c5be', 'request_id': 'req-badfa161-b4a8-44c6-9fcb-05eb661413f8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:32.283880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-badfa161-b4a8-44c6-9fcb-05eb661413f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63fb4664783f4667ba10d567e5c0c5be', 'request_id': 'req-badfa161-b4a8-44c6-9fcb-05eb661413f8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:32.293130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-badfa161-b4a8-44c6-9fcb-05eb661413f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63fb4664783f4667ba10d567e5c0c5be', 'request_id': 'req-badfa161-b4a8-44c6-9fcb-05eb661413f8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:32.308555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-badfa161-b4a8-44c6-9fcb-05eb661413f8 admin admin] The token cache is being invalidate because project 63fb4664783f4667ba10d567e5c0c5be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:32.309023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-badfa161-b4a8-44c6-9fcb-05eb661413f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:32.310654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-badfa161-b4a8-44c6-9fcb-05eb661413f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:32.320662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fd9df-a3fb-4229-af6f-19d1b10f280a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:32.321229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fd9df-a3fb-4229-af6f-19d1b10f280a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:32.321610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fd9df-a3fb-4229-af6f-19d1b10f280a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:32.325449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1000/1997] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:32 2022] DELETE /identity/v3/projects/63fb4664783f4667ba10d567e5c0c5be => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:32.358356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-185fd9df-a3fb-4229-af6f-19d1b10f280a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:32.360974 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-185fd9df-a3fb-4229-af6f-19d1b10f280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:32.483009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:32.509924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 998/1998] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:32.518424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c2fdc-0169-4cae-abb0-ad2a09715283 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:32.518813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c2fdc-0169-4cae-abb0-ad2a09715283 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:32.519223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c2fdc-0169-4cae-abb0-ad2a09715283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:32.544492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1a8c2fdc-0169-4cae-abb0-ad2a09715283 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:32.545759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1a8c2fdc-0169-4cae-abb0-ad2a09715283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:32.572169 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1001/1999] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:32.577729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fd9df-a3fb-4229-af6f-19d1b10f280a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:32.615731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fd9df-a3fb-4229-af6f-19d1b10f280a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:32.632782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26790ea8-3ebb-49f5-aee7-372dbcce4463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:32.633591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26790ea8-3ebb-49f5-aee7-372dbcce4463 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:32.633975 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26790ea8-3ebb-49f5-aee7-372dbcce4463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:32.635569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26790ea8-3ebb-49f5-aee7-372dbcce4463 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:32.637021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26790ea8-3ebb-49f5-aee7-372dbcce4463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:32.641415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26790ea8-3ebb-49f5-aee7-372dbcce4463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:32.656439 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26790ea8-3ebb-49f5-aee7-372dbcce4463 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:32.688682 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 999/2000] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:38:32.690527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c2fdc-0169-4cae-abb0-ad2a09715283 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:32.700718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c2fdc-0169-4cae-abb0-ad2a09715283 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:32.705959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:32.717035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5346c81-6895-40b2-8ab8-bbc9803971a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:32.718015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5346c81-6895-40b2-8ab8-bbc9803971a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:32.720707 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5346c81-6895-40b2-8ab8-bbc9803971a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:32.721278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5346c81-6895-40b2-8ab8-bbc9803971a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:32.723214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5346c81-6895-40b2-8ab8-bbc9803971a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:32.733798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5346c81-6895-40b2-8ab8-bbc9803971a2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:32.777097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1002/2001] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:32 2022] POST /identity/v3/projects => generated 365 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:32.778468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26790ea8-3ebb-49f5-aee7-372dbcce4463 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:32.795797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26790ea8-3ebb-49f5-aee7-372dbcce4463 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:32.871607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300f011-6f6f-4003-ad1d-bd54f566f547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:32.872250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300f011-6f6f-4003-ad1d-bd54f566f547 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:32.872535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300f011-6f6f-4003-ad1d-bd54f566f547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:32.872811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300f011-6f6f-4003-ad1d-bd54f566f547 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:32.873497 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300f011-6f6f-4003-ad1d-bd54f566f547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:32.875573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300f011-6f6f-4003-ad1d-bd54f566f547 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:32.915932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1000/2002] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:32 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:32.920043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5346c81-6895-40b2-8ab8-bbc9803971a2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:32.932727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5346c81-6895-40b2-8ab8-bbc9803971a2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:32.937650 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197c46a-9a97-48cf-a9a4-d4ccfa933b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:32.938573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197c46a-9a97-48cf-a9a4-d4ccfa933b10 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:32.939127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197c46a-9a97-48cf-a9a4-d4ccfa933b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:32.939482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197c46a-9a97-48cf-a9a4-d4ccfa933b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:32.941776 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197c46a-9a97-48cf-a9a4-d4ccfa933b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:32.943561 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197c46a-9a97-48cf-a9a4-d4ccfa933b10 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:32.955383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1003/2003] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:38:32.960056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300f011-6f6f-4003-ad1d-bd54f566f547 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:32.970884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300f011-6f6f-4003-ad1d-bd54f566f547 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:32.977354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e6f68-6f2c-4863-908e-94982d86a1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:32.978444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e6f68-6f2c-4863-908e-94982d86a1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:32.978906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e6f68-6f2c-4863-908e-94982d86a1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:32.979313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e6f68-6f2c-4863-908e-94982d86a1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:32.980348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e6f68-6f2c-4863-908e-94982d86a1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:32.982332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e6f68-6f2c-4863-908e-94982d86a1d6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:32.999678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1001/2004] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:33.001113 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197c46a-9a97-48cf-a9a4-d4ccfa933b10 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:33.023355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197c46a-9a97-48cf-a9a4-d4ccfa933b10 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:33.032037 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528e21-891f-4b09-8f42-5c015cf20d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:33.032788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528e21-891f-4b09-8f42-5c015cf20d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:33.033290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528e21-891f-4b09-8f42-5c015cf20d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:33.033571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528e21-891f-4b09-8f42-5c015cf20d5f admin admin] PATH_INFO: `/v3/projects/1f2047e7c9724b34bde99e63c07d7e8a/users/30f8f1c689544a4e87bb2080b5bd4759/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:33.056034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46528e21-891f-4b09-8f42-5c015cf20d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2047e7c9724b34bde99e63c07d7e8a, user_id=30f8f1c689544a4e87bb2080b5bd4759, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:33.059553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46528e21-891f-4b09-8f42-5c015cf20d5f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:33.075513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1004/2005] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:33 2022] PUT /identity/v3/projects/1f2047e7c9724b34bde99e63c07d7e8a/users/30f8f1c689544a4e87bb2080b5bd4759/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:38:33.089881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be750e-103d-43c4-9d0d-c36d322ac144 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:33.089881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be750e-103d-43c4-9d0d-c36d322ac144 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:33.089881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be750e-103d-43c4-9d0d-c36d322ac144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:33.127652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-39be750e-103d-43c4-9d0d-c36d322ac144 None None] MFA Rules not processed for user `30f8f1c689544a4e87bb2080b5bd4759`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:33.130106 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-39be750e-103d-43c4-9d0d-c36d322ac144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:33.254440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1002/2006] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:33.266314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528e21-891f-4b09-8f42-5c015cf20d5f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:33.269046 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-46528e21-891f-4b09-8f42-5c015cf20d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:33.384476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528e21-891f-4b09-8f42-5c015cf20d5f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:33.392451 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d80bf5-cb3a-4d07-a125-24b6a3c28990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:33.394190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d80bf5-cb3a-4d07-a125-24b6a3c28990 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:33.395295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d80bf5-cb3a-4d07-a125-24b6a3c28990 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:33.396188 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d80bf5-cb3a-4d07-a125-24b6a3c28990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:33.410149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d80bf5-cb3a-4d07-a125-24b6a3c28990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:33.412188 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d80bf5-cb3a-4d07-a125-24b6a3c28990 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:33.429851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1005/2007] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:38.365744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c0aa0-c04d-4d78-824a-4b464e95fdd4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:38.366363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c0aa0-c04d-4d78-824a-4b464e95fdd4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:38.366525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c0aa0-c04d-4d78-824a-4b464e95fdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:38.397790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dc5c0aa0-c04d-4d78-824a-4b464e95fdd4 None None] MFA Rules not processed for user `d714badcdf8e4461ae1f7ebe4f0c82d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:38.399012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dc5c0aa0-c04d-4d78-824a-4b464e95fdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:38.445003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:38.478301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1003/2008] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:38.483119 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d80bf5-cb3a-4d07-a125-24b6a3c28990 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:38.484711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-67d80bf5-cb3a-4d07-a125-24b6a3c28990 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:38.591291 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d80bf5-cb3a-4d07-a125-24b6a3c28990 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:38.612836 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425687b2-1cf2-4964-92fe-5c1f98d99caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:38.613655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425687b2-1cf2-4964-92fe-5c1f98d99caa admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:38.614094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425687b2-1cf2-4964-92fe-5c1f98d99caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:38.614534 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425687b2-1cf2-4964-92fe-5c1f98d99caa admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:38.615393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425687b2-1cf2-4964-92fe-5c1f98d99caa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:38.620847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425687b2-1cf2-4964-92fe-5c1f98d99caa admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:38.666020 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1006/2009] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:38 2022] POST /identity/v3/projects => generated 371 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:38.669776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c0aa0-c04d-4d78-824a-4b464e95fdd4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:38.682896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c0aa0-c04d-4d78-824a-4b464e95fdd4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:38.795059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c7108-a01b-47a1-945c-7b197c78e51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:38.795899 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c7108-a01b-47a1-945c-7b197c78e51c admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:38.796263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c7108-a01b-47a1-945c-7b197c78e51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:38.797466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c7108-a01b-47a1-945c-7b197c78e51c admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:38.798403 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c7108-a01b-47a1-945c-7b197c78e51c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:38.801084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c7108-a01b-47a1-945c-7b197c78e51c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:38.836199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1004/2010] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:38 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:38.839885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425687b2-1cf2-4964-92fe-5c1f98d99caa admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:38.858717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425687b2-1cf2-4964-92fe-5c1f98d99caa admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:38.869662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060a5bd-6d21-48c4-b141-e15c22f577eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:38.870768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060a5bd-6d21-48c4-b141-e15c22f577eb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:38.871218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060a5bd-6d21-48c4-b141-e15c22f577eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:38.871582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060a5bd-6d21-48c4-b141-e15c22f577eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:38.872504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060a5bd-6d21-48c4-b141-e15c22f577eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:38.874559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060a5bd-6d21-48c4-b141-e15c22f577eb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:38.892100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1007/2011] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:38.896571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c7108-a01b-47a1-945c-7b197c78e51c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:38.916337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c7108-a01b-47a1-945c-7b197c78e51c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:38.923721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b7bc3-95a9-44bb-8f42-044f774d06b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:38.925174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b7bc3-95a9-44bb-8f42-044f774d06b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:38.925582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b7bc3-95a9-44bb-8f42-044f774d06b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:38.925980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b7bc3-95a9-44bb-8f42-044f774d06b9 admin admin] PATH_INFO: `/v3/projects/c8c26cad6da4415a97e1816d8da3cfef/users/9b9897594a6c4aafbb58e94d1caf9b52/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:38.969423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586b7bc3-95a9-44bb-8f42-044f774d06b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c26cad6da4415a97e1816d8da3cfef, user_id=9b9897594a6c4aafbb58e94d1caf9b52, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:38.973245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586b7bc3-95a9-44bb-8f42-044f774d06b9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:38.999496 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1005/2012] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:38 2022] PUT /identity/v3/projects/c8c26cad6da4415a97e1816d8da3cfef/users/9b9897594a6c4aafbb58e94d1caf9b52/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:38.999987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060a5bd-6d21-48c4-b141-e15c22f577eb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:39.017133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060a5bd-6d21-48c4-b141-e15c22f577eb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:39.045374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:39.104684 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa3b04-0098-4b7a-aac0-a3c4e8a2a58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:39.105314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa3b04-0098-4b7a-aac0-a3c4e8a2a58e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:39.105605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa3b04-0098-4b7a-aac0-a3c4e8a2a58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:39.105944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa3b04-0098-4b7a-aac0-a3c4e8a2a58e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:39.106643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa3b04-0098-4b7a-aac0-a3c4e8a2a58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:39.108571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa3b04-0098-4b7a-aac0-a3c4e8a2a58e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:39.121317 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1008/2013] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:39.125430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b7bc3-95a9-44bb-8f42-044f774d06b9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:39.136576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b7bc3-95a9-44bb-8f42-044f774d06b9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:39.141888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a27d0c-fdf5-4cb5-9e2e-2d69bcb831f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:39.142629 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a27d0c-fdf5-4cb5-9e2e-2d69bcb831f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:39.142909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a27d0c-fdf5-4cb5-9e2e-2d69bcb831f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:39.143373 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a27d0c-fdf5-4cb5-9e2e-2d69bcb831f5 admin admin] PATH_INFO: `/v3/projects/c8c26cad6da4415a97e1816d8da3cfef/users/9b9897594a6c4aafbb58e94d1caf9b52/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:39.147291 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a27d0c-fdf5-4cb5-9e2e-2d69bcb831f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c26cad6da4415a97e1816d8da3cfef, user_id=9b9897594a6c4aafbb58e94d1caf9b52, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:39.149938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a27d0c-fdf5-4cb5-9e2e-2d69bcb831f5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:39.163742 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1006/2014] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:39 2022] PUT /identity/v3/projects/c8c26cad6da4415a97e1816d8da3cfef/users/9b9897594a6c4aafbb58e94d1caf9b52/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:43.250108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa3b04-0098-4b7a-aac0-a3c4e8a2a58e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:43.252083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f5fa3b04-0098-4b7a-aac0-a3c4e8a2a58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:43.272809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa3b04-0098-4b7a-aac0-a3c4e8a2a58e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:43.379877 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c777c-7601-47a8-a6a4-49a69e77b218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:43.380550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c777c-7601-47a8-a6a4-49a69e77b218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:43.380841 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c777c-7601-47a8-a6a4-49a69e77b218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:43.381129 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c777c-7601-47a8-a6a4-49a69e77b218 admin admin] PATH_INFO: `/v3/users/905e317fcf49428ea1fa708dfabc958e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:43.382354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c777c-7601-47a8-a6a4-49a69e77b218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=905e317fcf49428ea1fa708dfabc958e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:43.385021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c777c-7601-47a8-a6a4-49a69e77b218 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:43.469504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-809c777c-7601-47a8-a6a4-49a69e77b218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '905e317fcf49428ea1fa708dfabc958e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:43.479543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-809c777c-7601-47a8-a6a4-49a69e77b218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '905e317fcf49428ea1fa708dfabc958e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:43.497768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-809c777c-7601-47a8-a6a4-49a69e77b218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '905e317fcf49428ea1fa708dfabc958e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:43.498959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-809c777c-7601-47a8-a6a4-49a69e77b218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '905e317fcf49428ea1fa708dfabc958e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:43.499662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-809c777c-7601-47a8-a6a4-49a69e77b218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '905e317fcf49428ea1fa708dfabc958e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:43.515579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1009/2015] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:43 2022] DELETE /identity/v3/users/905e317fcf49428ea1fa708dfabc958e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:43.831772 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a27d0c-fdf5-4cb5-9e2e-2d69bcb831f5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:43.834134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-59a27d0c-fdf5-4cb5-9e2e-2d69bcb831f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:43.856291 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a27d0c-fdf5-4cb5-9e2e-2d69bcb831f5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:43.991714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60daa69-bd15-492e-b2e1-3349ff10d000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:43.992448 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60daa69-bd15-492e-b2e1-3349ff10d000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:43.992815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60daa69-bd15-492e-b2e1-3349ff10d000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:43.993159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60daa69-bd15-492e-b2e1-3349ff10d000 admin admin] PATH_INFO: `/v3/projects/53bab4dbf0c34ae0be1be397427c99e8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:43.996867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60daa69-bd15-492e-b2e1-3349ff10d000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53bab4dbf0c34ae0be1be397427c99e8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:43.998895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60daa69-bd15-492e-b2e1-3349ff10d000 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:44.062051 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:44.081617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c60daa69-bd15-492e-b2e1-3349ff10d000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53bab4dbf0c34ae0be1be397427c99e8', 'request_id': 'req-c60daa69-bd15-492e-b2e1-3349ff10d000'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:44.091310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c60daa69-bd15-492e-b2e1-3349ff10d000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53bab4dbf0c34ae0be1be397427c99e8', 'request_id': 'req-c60daa69-bd15-492e-b2e1-3349ff10d000'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:44.103500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c60daa69-bd15-492e-b2e1-3349ff10d000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53bab4dbf0c34ae0be1be397427c99e8', 'request_id': 'req-c60daa69-bd15-492e-b2e1-3349ff10d000'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:44.131879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c60daa69-bd15-492e-b2e1-3349ff10d000 admin admin] The token cache is being invalidate because project 53bab4dbf0c34ae0be1be397427c99e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:44.132299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c60daa69-bd15-492e-b2e1-3349ff10d000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:44.132949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c60daa69-bd15-492e-b2e1-3349ff10d000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:44.135162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1007/2016] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:43 2022] DELETE /identity/v3/projects/53bab4dbf0c34ae0be1be397427c99e8 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:44.140809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c26479-66a1-4ce2-8747-4c0c2533c62c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:44.141498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c26479-66a1-4ce2-8747-4c0c2533c62c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:44.142035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c26479-66a1-4ce2-8747-4c0c2533c62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:44.168503 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b9c26479-66a1-4ce2-8747-4c0c2533c62c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:44.170020 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b9c26479-66a1-4ce2-8747-4c0c2533c62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:44.215530 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:44.259373 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1010/2017] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:44.266922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e96ca-f12e-4938-953a-a8e04147d7b2 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:44.267243 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e96ca-f12e-4938-953a-a8e04147d7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:44.267527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e96ca-f12e-4938-953a-a8e04147d7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:44.290808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f10e96ca-f12e-4938-953a-a8e04147d7b2 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:44.292090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f10e96ca-f12e-4938-953a-a8e04147d7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:44.302064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1008/2018] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:44.306297 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c26479-66a1-4ce2-8747-4c0c2533c62c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:44.317240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c26479-66a1-4ce2-8747-4c0c2533c62c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:44.322710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611182f-6335-4a19-ba49-533b0e5e9e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:44.323649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611182f-6335-4a19-ba49-533b0e5e9e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:44.324766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611182f-6335-4a19-ba49-533b0e5e9e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:44.324766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611182f-6335-4a19-ba49-533b0e5e9e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:44.324950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611182f-6335-4a19-ba49-533b0e5e9e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:44.325241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611182f-6335-4a19-ba49-533b0e5e9e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:44.326854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611182f-6335-4a19-ba49-533b0e5e9e8b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:44.344325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1011/2019] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:38:44.346753 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e96ca-f12e-4938-953a-a8e04147d7b2 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:44.357444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e96ca-f12e-4938-953a-a8e04147d7b2 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:44.363794 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704fbcb-608c-47dc-bcc6-76b798fe78c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:44.364430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704fbcb-608c-47dc-bcc6-76b798fe78c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:44.365017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704fbcb-608c-47dc-bcc6-76b798fe78c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:44.365017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704fbcb-608c-47dc-bcc6-76b798fe78c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:44.365643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704fbcb-608c-47dc-bcc6-76b798fe78c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:44.367591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704fbcb-608c-47dc-bcc6-76b798fe78c9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:44.404607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1009/2020] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:44 2022] POST /identity/v3/projects => generated 403 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:44.406037 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611182f-6335-4a19-ba49-533b0e5e9e8b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:44.417650 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611182f-6335-4a19-ba49-533b0e5e9e8b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:44.530704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323acde-d69e-4298-858e-4844cfe88ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:44.531712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323acde-d69e-4298-858e-4844cfe88ded admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:44.532192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323acde-d69e-4298-858e-4844cfe88ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:44.532652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323acde-d69e-4298-858e-4844cfe88ded admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:44.533968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323acde-d69e-4298-858e-4844cfe88ded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:44.536714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323acde-d69e-4298-858e-4844cfe88ded admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:44.580145 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1012/2021] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:44 2022] POST /identity/v3/users => generated 302 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:44.584943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704fbcb-608c-47dc-bcc6-76b798fe78c9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:44.601054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704fbcb-608c-47dc-bcc6-76b798fe78c9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:44.618131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57fa42-549c-450f-bfe1-8bfde7874b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:44.619732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57fa42-549c-450f-bfe1-8bfde7874b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:44.620031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57fa42-549c-450f-bfe1-8bfde7874b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:44.620317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57fa42-549c-450f-bfe1-8bfde7874b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:44.621033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57fa42-549c-450f-bfe1-8bfde7874b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:44.622889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57fa42-549c-450f-bfe1-8bfde7874b44 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:44.635475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1010/2022] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:38:44.639378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323acde-d69e-4298-858e-4844cfe88ded admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:44.650842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323acde-d69e-4298-858e-4844cfe88ded admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:44.657086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03102c4-062a-4514-8468-4c1be04ed11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:44.657724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03102c4-062a-4514-8468-4c1be04ed11e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:44.658018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03102c4-062a-4514-8468-4c1be04ed11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:44.658302 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03102c4-062a-4514-8468-4c1be04ed11e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:44.659103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03102c4-062a-4514-8468-4c1be04ed11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:44.660849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03102c4-062a-4514-8468-4c1be04ed11e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:44.672408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1013/2023] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:44.676497 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57fa42-549c-450f-bfe1-8bfde7874b44 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:44.690793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57fa42-549c-450f-bfe1-8bfde7874b44 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:44.695971 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f877c40-d6d9-4e81-a029-f0189ec9763c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:44.696811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f877c40-d6d9-4e81-a029-f0189ec9763c admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:44.697110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f877c40-d6d9-4e81-a029-f0189ec9763c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:44.697399 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f877c40-d6d9-4e81-a029-f0189ec9763c admin admin] PATH_INFO: `/v3/projects/06a52e710a2d4078a18dbade53b9145a/users/6199255d28524576bb817d85846b52ac/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:44.721352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f877c40-d6d9-4e81-a029-f0189ec9763c admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a52e710a2d4078a18dbade53b9145a, user_id=6199255d28524576bb817d85846b52ac, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:44.724231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f877c40-d6d9-4e81-a029-f0189ec9763c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:44.739547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1011/2024] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:44 2022] PUT /identity/v3/projects/06a52e710a2d4078a18dbade53b9145a/users/6199255d28524576bb817d85846b52ac/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:38:44.750246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df923611-0abe-455c-a6b5-c2f4aa163e03 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:44.750908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df923611-0abe-455c-a6b5-c2f4aa163e03 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:44.750908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df923611-0abe-455c-a6b5-c2f4aa163e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:44.800402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-df923611-0abe-455c-a6b5-c2f4aa163e03 None None] MFA Rules not processed for user `6199255d28524576bb817d85846b52ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:44.801958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-df923611-0abe-455c-a6b5-c2f4aa163e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:44.890381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1014/2025] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:44.892358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f877c40-d6d9-4e81-a029-f0189ec9763c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:44.989048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f877c40-d6d9-4e81-a029-f0189ec9763c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:44.995000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4007e-d8ea-4f0f-9799-4fc9b9c92f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:44.996121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4007e-d8ea-4f0f-9799-4fc9b9c92f18 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:44.996404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4007e-d8ea-4f0f-9799-4fc9b9c92f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:44.996686 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4007e-d8ea-4f0f-9799-4fc9b9c92f18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:44.997387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4007e-d8ea-4f0f-9799-4fc9b9c92f18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:44.999522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4007e-d8ea-4f0f-9799-4fc9b9c92f18 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:45.049880 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1012/2026] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:44 2022] POST /identity/v3/projects => generated 403 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:45.053596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df923611-0abe-455c-a6b5-c2f4aa163e03 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:45.079955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df923611-0abe-455c-a6b5-c2f4aa163e03 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:45.146613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:45.248333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa679b7-9a07-469e-adc8-1060a41d8d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:45.249102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa679b7-9a07-469e-adc8-1060a41d8d1d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:45.249445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa679b7-9a07-469e-adc8-1060a41d8d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:45.249840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa679b7-9a07-469e-adc8-1060a41d8d1d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:45.250786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa679b7-9a07-469e-adc8-1060a41d8d1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:45.256558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa679b7-9a07-469e-adc8-1060a41d8d1d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:45.307122 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1015/2027] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:45 2022] POST /identity/v3/users => generated 308 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:45.309758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4007e-d8ea-4f0f-9799-4fc9b9c92f18 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:45.325424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4007e-d8ea-4f0f-9799-4fc9b9c92f18 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:45.331294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237607a-024d-4f4d-b9b7-a728ef2845ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:45.331940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237607a-024d-4f4d-b9b7-a728ef2845ba admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:45.332227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237607a-024d-4f4d-b9b7-a728ef2845ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:45.332781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237607a-024d-4f4d-b9b7-a728ef2845ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:45.333215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237607a-024d-4f4d-b9b7-a728ef2845ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:45.334846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237607a-024d-4f4d-b9b7-a728ef2845ba admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:45.350028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1013/2028] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:45.354186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa679b7-9a07-469e-adc8-1060a41d8d1d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:45.367891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa679b7-9a07-469e-adc8-1060a41d8d1d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:45.373912 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e33199-1240-44e1-9d92-7e47e17fcafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:45.374284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33199-1240-44e1-9d92-7e47e17fcafa admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:45.374731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33199-1240-44e1-9d92-7e47e17fcafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:45.375722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33199-1240-44e1-9d92-7e47e17fcafa admin admin] PATH_INFO: `/v3/projects/75ad88728b1c48e49024ff95be1815c2/users/7baeea0ff20648fd8ef8783df9617c6a/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:45.395393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e33199-1240-44e1-9d92-7e47e17fcafa admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ad88728b1c48e49024ff95be1815c2, user_id=7baeea0ff20648fd8ef8783df9617c6a, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:45.398457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e33199-1240-44e1-9d92-7e47e17fcafa admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:45.425599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1016/2029] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:45 2022] PUT /identity/v3/projects/75ad88728b1c48e49024ff95be1815c2/users/7baeea0ff20648fd8ef8783df9617c6a/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:38:45.426782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237607a-024d-4f4d-b9b7-a728ef2845ba admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:45.438727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237607a-024d-4f4d-b9b7-a728ef2845ba admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:45.492436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:45.572103 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53232eab-57f2-45ef-9554-9d98f6529106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:45.573045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53232eab-57f2-45ef-9554-9d98f6529106 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:45.573501 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53232eab-57f2-45ef-9554-9d98f6529106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:45.575155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53232eab-57f2-45ef-9554-9d98f6529106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:45.577121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53232eab-57f2-45ef-9554-9d98f6529106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:45.579217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53232eab-57f2-45ef-9554-9d98f6529106 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:45.599666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1014/2030] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:45.603302 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e33199-1240-44e1-9d92-7e47e17fcafa admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:45.625582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e33199-1240-44e1-9d92-7e47e17fcafa admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:45.631616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadeee7-c63e-4692-ba64-4c7f871c06bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:45.633251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadeee7-c63e-4692-ba64-4c7f871c06bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:45.633251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadeee7-c63e-4692-ba64-4c7f871c06bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:45.633654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadeee7-c63e-4692-ba64-4c7f871c06bf admin admin] PATH_INFO: `/v3/projects/75ad88728b1c48e49024ff95be1815c2/users/7baeea0ff20648fd8ef8783df9617c6a/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:45.635883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdadeee7-c63e-4692-ba64-4c7f871c06bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ad88728b1c48e49024ff95be1815c2, user_id=7baeea0ff20648fd8ef8783df9617c6a, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:45.638713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdadeee7-c63e-4692-ba64-4c7f871c06bf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:45.653103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1017/2031] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:45 2022] PUT /identity/v3/projects/75ad88728b1c48e49024ff95be1815c2/users/7baeea0ff20648fd8ef8783df9617c6a/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:45.666249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49ab46-b53c-499c-865f-9808095c47a6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:45.666584 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49ab46-b53c-499c-865f-9808095c47a6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:45.666887 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49ab46-b53c-499c-865f-9808095c47a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:45.700702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0f49ab46-b53c-499c-865f-9808095c47a6 None None] MFA Rules not processed for user `7baeea0ff20648fd8ef8783df9617c6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:45.701947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0f49ab46-b53c-499c-865f-9808095c47a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:45.789885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1015/2032] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:45 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:45.800890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadeee7-c63e-4692-ba64-4c7f871c06bf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:45.803155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fdadeee7-c63e-4692-ba64-4c7f871c06bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:45.890311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadeee7-c63e-4692-ba64-4c7f871c06bf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:45.895939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871f77e-9659-471e-a19d-4d5182ff1d2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:45.896718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871f77e-9659-471e-a19d-4d5182ff1d2a service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:45.897494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871f77e-9659-471e-a19d-4d5182ff1d2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:45.898476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871f77e-9659-471e-a19d-4d5182ff1d2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:45.912786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871f77e-9659-471e-a19d-4d5182ff1d2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:45.915090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871f77e-9659-471e-a19d-4d5182ff1d2a service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:45.933993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1018/2033] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:45 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:46.162184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49ab46-b53c-499c-865f-9808095c47a6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:46.238584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f306eb-cd88-4369-bb77-208ffba9114b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:46.238882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f306eb-cd88-4369-bb77-208ffba9114b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:46.239851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f306eb-cd88-4369-bb77-208ffba9114b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:46.272228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49ab46-b53c-499c-865f-9808095c47a6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:46.280985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-61f306eb-cd88-4369-bb77-208ffba9114b None None] MFA Rules not processed for user `9b9897594a6c4aafbb58e94d1caf9b52`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:46.282554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-61f306eb-cd88-4369-bb77-208ffba9114b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:46.292834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:46.293642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:46.293942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:46.294220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] PATH_INFO: `/v3/users/6199255d28524576bb817d85846b52ac` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:46.295445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6199255d28524576bb817d85846b52ac)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:46.297418 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:46.421526 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6199255d28524576bb817d85846b52ac'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:46.432140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:46.439038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6199255d28524576bb817d85846b52ac'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:46.440191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6199255d28524576bb817d85846b52ac'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:46.450059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6199255d28524576bb817d85846b52ac'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:46.461576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6199255d28524576bb817d85846b52ac'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:46.474978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1019/2035] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:46 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:46.477067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ef3e8-3908-4971-bf95-4413c6d97d45 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:46.477566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ef3e8-3908-4971-bf95-4413c6d97d45 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:46.478093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ef3e8-3908-4971-bf95-4413c6d97d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:46.483165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1016/2035] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:46 2022] DELETE /identity/v3/users/6199255d28524576bb817d85846b52ac => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:46.501199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ae0ef3e8-3908-4971-bf95-4413c6d97d45 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:46.507155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ae0ef3e8-3908-4971-bf95-4413c6d97d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:46.527725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:46.561927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:46.569442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:46.662202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1020/2036] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:46.672388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ef3e8-3908-4971-bf95-4413c6d97d45 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:46.673875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ae0ef3e8-3908-4971-bf95-4413c6d97d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:46.724001 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cde629-4316-4342-81bd-bb1d73a3cea2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:46.733052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d900d3-7614-4997-80d7-e915d5486f59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:46.733052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d900d3-7614-4997-80d7-e915d5486f59 service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:46.733052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d900d3-7614-4997-80d7-e915d5486f59 service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:46.733572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d900d3-7614-4997-80d7-e915d5486f59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:46.735524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-24d900d3-7614-4997-80d7-e915d5486f59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:46.814590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ef3e8-3908-4971-bf95-4413c6d97d45 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:46.822566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a92e6-184d-4d70-97ab-dd1077f6757d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:46.822566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a92e6-184d-4d70-97ab-dd1077f6757d service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:46.822566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a92e6-184d-4d70-97ab-dd1077f6757d service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:46.822915 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a92e6-184d-4d70-97ab-dd1077f6757d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:46.825734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d900d3-7614-4997-80d7-e915d5486f59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:46.827708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d900d3-7614-4997-80d7-e915d5486f59 service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:46.835756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a92e6-184d-4d70-97ab-dd1077f6757d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:46.838331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a92e6-184d-4d70-97ab-dd1077f6757d service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:46.839704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1017/2037] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:46.872158 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1021/2038] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:38:47.282839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d900d3-7614-4997-80d7-e915d5486f59 service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:47.286461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-24d900d3-7614-4997-80d7-e915d5486f59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:47.311445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d900d3-7614-4997-80d7-e915d5486f59 service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:47.322961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:47.323672 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:47.324135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:47.324309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd admin admin] PATH_INFO: `/v3/projects/06a52e710a2d4078a18dbade53b9145a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:47.327923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a52e710a2d4078a18dbade53b9145a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:47.329884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:47.397065 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a52e710a2d4078a18dbade53b9145a', 'request_id': 'req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:47.404542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a52e710a2d4078a18dbade53b9145a', 'request_id': 'req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:47.412290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a52e710a2d4078a18dbade53b9145a', 'request_id': 'req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:47.415650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:47.431477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd admin admin] The token cache is being invalidate because project 06a52e710a2d4078a18dbade53b9145a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:47.432073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:47.433089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0ddf5e99-7cfd-47cd-b50a-7c92dc9b3efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:47.435270 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1018/2039] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:47 2022] DELETE /identity/v3/projects/06a52e710a2d4078a18dbade53b9145a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:47.439024 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a92e6-184d-4d70-97ab-dd1077f6757d service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:47.450786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a92e6-184d-4d70-97ab-dd1077f6757d service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:47.599699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:47.601133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:47.601358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:47.602022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] PATH_INFO: `/v3/users/7baeea0ff20648fd8ef8783df9617c6a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:47.604520 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7baeea0ff20648fd8ef8783df9617c6a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:47.606538 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:47.646424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4d1be-045f-415b-a802-b498a0f43a41 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:47.646716 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4d1be-045f-415b-a802-b498a0f43a41 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:47.647153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4d1be-045f-415b-a802-b498a0f43a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:47.682079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-feb4d1be-045f-415b-a802-b498a0f43a41 None None] MFA Rules not processed for user `9255edea8b504727a587443e124f3e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:47.684443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-feb4d1be-045f-415b-a802-b498a0f43a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:47.729500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7baeea0ff20648fd8ef8783df9617c6a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:47.743213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7baeea0ff20648fd8ef8783df9617c6a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:47.751308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:47.771679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7baeea0ff20648fd8ef8783df9617c6a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:47.772837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7baeea0ff20648fd8ef8783df9617c6a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:47.773973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7baeea0ff20648fd8ef8783df9617c6a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:47.792910 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1022/2040] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:47 2022] DELETE /identity/v3/users/7baeea0ff20648fd8ef8783df9617c6a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:47.803726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1019/2041] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:48.262345 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:48.264431 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:48.357968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b75a0b-b684-4bf8-84cd-47e15c92fbbb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:48.364527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:48.365206 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:48.365500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:48.365774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c admin admin] PATH_INFO: `/v3/projects/75ad88728b1c48e49024ff95be1815c2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:48.366966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=75ad88728b1c48e49024ff95be1815c2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:48.368983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:48.424937 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75ad88728b1c48e49024ff95be1815c2', 'request_id': 'req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:48.434396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75ad88728b1c48e49024ff95be1815c2', 'request_id': 'req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:48.442286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75ad88728b1c48e49024ff95be1815c2', 'request_id': 'req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:48.457281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c admin admin] The token cache is being invalidate because project 75ad88728b1c48e49024ff95be1815c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:48.457610 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:48.458493 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47b2c5c4-4d9c-464b-ba5a-8ccb2bf0eb9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:48.460704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1023/2042] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:48 2022] DELETE /identity/v3/projects/75ad88728b1c48e49024ff95be1815c2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:48.465504 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb72f1-1948-4ffc-b505-44765b2a7482 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:48.465960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb72f1-1948-4ffc-b505-44765b2a7482 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:48.466448 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb72f1-1948-4ffc-b505-44765b2a7482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:48.491021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b4cb72f1-1948-4ffc-b505-44765b2a7482 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:48.492619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b4cb72f1-1948-4ffc-b505-44765b2a7482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:48.587962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e80d9-4690-450e-8c19-7adcbcaf0ce5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:48.588250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e80d9-4690-450e-8c19-7adcbcaf0ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:48.588530 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e80d9-4690-450e-8c19-7adcbcaf0ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:48.595505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1020/2043] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:48.605286 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe299f-f81b-462b-bb46-6e8843392107 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:48.605580 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe299f-f81b-462b-bb46-6e8843392107 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:48.606077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe299f-f81b-462b-bb46-6e8843392107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:48.628755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e47e80d9-4690-450e-8c19-7adcbcaf0ce5 None None] MFA Rules not processed for user `d714badcdf8e4461ae1f7ebe4f0c82d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:48.630352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e47e80d9-4690-450e-8c19-7adcbcaf0ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:48.634215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7cfe299f-f81b-462b-bb46-6e8843392107 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:48.637337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7cfe299f-f81b-462b-bb46-6e8843392107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:48.653027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1021/2044] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:48.657180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe299f-f81b-462b-bb46-6e8843392107 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:48.685732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe299f-f81b-462b-bb46-6e8843392107 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:48.695454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3e440-1212-4b48-9a84-1ece4c1512ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:48.696089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3e440-1212-4b48-9a84-1ece4c1512ae admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:48.696374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3e440-1212-4b48-9a84-1ece4c1512ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:48.696652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3e440-1212-4b48-9a84-1ece4c1512ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:48.698601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3e440-1212-4b48-9a84-1ece4c1512ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:48.698601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3e440-1212-4b48-9a84-1ece4c1512ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:48.701179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3e440-1212-4b48-9a84-1ece4c1512ae admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:48.724107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1022/2045] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:38:48.725211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3e440-1212-4b48-9a84-1ece4c1512ae admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:48.743636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3e440-1212-4b48-9a84-1ece4c1512ae admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:48.750743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf29e2-ef5b-43e0-933d-0bc74a5f5915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:48.751425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf29e2-ef5b-43e0-933d-0bc74a5f5915 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:48.751863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf29e2-ef5b-43e0-933d-0bc74a5f5915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:48.752345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf29e2-ef5b-43e0-933d-0bc74a5f5915 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:48.753684 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf29e2-ef5b-43e0-933d-0bc74a5f5915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:48.756665 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf29e2-ef5b-43e0-933d-0bc74a5f5915 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:48.777377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1024/2046] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:48.806047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1023/2047] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:48 2022] POST /identity/v3/projects => generated 367 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:48.807336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e80d9-4690-450e-8c19-7adcbcaf0ce5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:48.826444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e80d9-4690-450e-8c19-7adcbcaf0ce5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:48.843682 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:48.943549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def2a75-ace2-4ba5-8446-f82311adf224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:48.943549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def2a75-ace2-4ba5-8446-f82311adf224 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:48.943549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def2a75-ace2-4ba5-8446-f82311adf224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:48.943549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def2a75-ace2-4ba5-8446-f82311adf224 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:48.944064 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def2a75-ace2-4ba5-8446-f82311adf224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:48.947117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def2a75-ace2-4ba5-8446-f82311adf224 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:48.979263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1025/2048] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:48 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:48.983146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf29e2-ef5b-43e0-933d-0bc74a5f5915 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:48.995624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf29e2-ef5b-43e0-933d-0bc74a5f5915 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:49.003086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a9234-48a2-4e7f-9f13-0c90a732f0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:49.004011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9234-48a2-4e7f-9f13-0c90a732f0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:49.004308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9234-48a2-4e7f-9f13-0c90a732f0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:49.005161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9234-48a2-4e7f-9f13-0c90a732f0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:49.005884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a9234-48a2-4e7f-9f13-0c90a732f0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:49.007631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a9234-48a2-4e7f-9f13-0c90a732f0f8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:49.030329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1024/2049] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:49.031646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def2a75-ace2-4ba5-8446-f82311adf224 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:49.042613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def2a75-ace2-4ba5-8446-f82311adf224 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:49.050653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f2f8e-a722-43ab-92f5-a50de59cfd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:49.052034 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f2f8e-a722-43ab-92f5-a50de59cfd5c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:49.052716 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f2f8e-a722-43ab-92f5-a50de59cfd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:49.053187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f2f8e-a722-43ab-92f5-a50de59cfd5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:49.054652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f2f8e-a722-43ab-92f5-a50de59cfd5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:49.057163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f2f8e-a722-43ab-92f5-a50de59cfd5c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:49.082831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1026/2050] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:49.087274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a9234-48a2-4e7f-9f13-0c90a732f0f8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:49.100557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a9234-48a2-4e7f-9f13-0c90a732f0f8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:49.106630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4afca-23a4-431a-b231-fb8c33439725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:49.107396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4afca-23a4-431a-b231-fb8c33439725 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:49.107674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4afca-23a4-431a-b231-fb8c33439725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:49.108188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4afca-23a4-431a-b231-fb8c33439725 admin admin] PATH_INFO: `/v3/projects/6951ba89bce243b484e35fdab09036e1/users/26084bb188d34d1e9647e4964e839e70/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:49.132726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4afca-23a4-431a-b231-fb8c33439725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6951ba89bce243b484e35fdab09036e1, user_id=26084bb188d34d1e9647e4964e839e70, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:49.138529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4afca-23a4-431a-b231-fb8c33439725 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:49.159518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1025/2051] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:49 2022] PUT /identity/v3/projects/6951ba89bce243b484e35fdab09036e1/users/26084bb188d34d1e9647e4964e839e70/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:49.171231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d5bfe-ed43-4f69-8eab-f671b6cfa129 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:49.171738 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d5bfe-ed43-4f69-8eab-f671b6cfa129 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:49.172577 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d5bfe-ed43-4f69-8eab-f671b6cfa129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:49.233922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d77d5bfe-ed43-4f69-8eab-f671b6cfa129 None None] MFA Rules not processed for user `26084bb188d34d1e9647e4964e839e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:49.235277 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d77d5bfe-ed43-4f69-8eab-f671b6cfa129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:49.326326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1027/2052] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:49.331970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4afca-23a4-431a-b231-fb8c33439725 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:49.375371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:49.462544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4afca-23a4-431a-b231-fb8c33439725 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:49.469308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b2953-fa43-4fd1-8db1-05c2ff9264f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:49.470199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b2953-fa43-4fd1-8db1-05c2ff9264f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:49.471893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b2953-fa43-4fd1-8db1-05c2ff9264f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:49.472781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b2953-fa43-4fd1-8db1-05c2ff9264f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:49.474286 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b2953-fa43-4fd1-8db1-05c2ff9264f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:49.478061 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b2953-fa43-4fd1-8db1-05c2ff9264f8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:49.518603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1026/2053] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:49 2022] POST /identity/v3/projects => generated 369 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:49.519919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d5bfe-ed43-4f69-8eab-f671b6cfa129 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:49.543127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d5bfe-ed43-4f69-8eab-f671b6cfa129 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:49.625235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffc004-d797-4b34-8dcf-77a593418417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:49.626227 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffc004-d797-4b34-8dcf-77a593418417 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:49.626542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffc004-d797-4b34-8dcf-77a593418417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:49.626896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffc004-d797-4b34-8dcf-77a593418417 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:49.628994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffc004-d797-4b34-8dcf-77a593418417 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:49.638223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffc004-d797-4b34-8dcf-77a593418417 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:49.667814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1028/2054] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:49 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:38:49.671466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b2953-fa43-4fd1-8db1-05c2ff9264f8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:49.694515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b2953-fa43-4fd1-8db1-05c2ff9264f8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:49.700579 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b3956-1fbe-446f-9102-b3b7d76d8a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:49.701632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b3956-1fbe-446f-9102-b3b7d76d8a6e admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:49.702066 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b3956-1fbe-446f-9102-b3b7d76d8a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:49.702636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b3956-1fbe-446f-9102-b3b7d76d8a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:49.703643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b3956-1fbe-446f-9102-b3b7d76d8a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:49.706276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b3956-1fbe-446f-9102-b3b7d76d8a6e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:49.722549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1027/2055] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:49.736954 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffc004-d797-4b34-8dcf-77a593418417 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:49.750027 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffc004-d797-4b34-8dcf-77a593418417 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:49.755939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dff4e-d09f-4b8c-aed0-919a8f119455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:49.756955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dff4e-d09f-4b8c-aed0-919a8f119455 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:49.758091 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dff4e-d09f-4b8c-aed0-919a8f119455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:49.758470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dff4e-d09f-4b8c-aed0-919a8f119455 admin admin] PATH_INFO: `/v3/projects/a5881c187afc4c4fb48210a95bb047cf/users/1a14627b63ea49f5aa251d460b5da4bf/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:49.763403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:49.806644 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7dff4e-d09f-4b8c-aed0-919a8f119455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5881c187afc4c4fb48210a95bb047cf, user_id=1a14627b63ea49f5aa251d460b5da4bf, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:49.821581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7dff4e-d09f-4b8c-aed0-919a8f119455 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:49.851599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1029/2056] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:49 2022] PUT /identity/v3/projects/a5881c187afc4c4fb48210a95bb047cf/users/1a14627b63ea49f5aa251d460b5da4bf/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:49.856347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b3956-1fbe-446f-9102-b3b7d76d8a6e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:49.881033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b3956-1fbe-446f-9102-b3b7d76d8a6e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:49.977055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e8082-14f4-4846-9468-9a3c3007bd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:49.977759 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8082-14f4-4846-9468-9a3c3007bd6c admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:49.978149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8082-14f4-4846-9468-9a3c3007bd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:49.978527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8082-14f4-4846-9468-9a3c3007bd6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:49.979377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e8082-14f4-4846-9468-9a3c3007bd6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:49.982578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e8082-14f4-4846-9468-9a3c3007bd6c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:50.009655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1028/2057] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:50.013807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dff4e-d09f-4b8c-aed0-919a8f119455 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:50.027553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dff4e-d09f-4b8c-aed0-919a8f119455 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:50.053303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68516074-c517-4f01-9136-6cc840eb369c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:50.054183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68516074-c517-4f01-9136-6cc840eb369c admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:50.056038 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68516074-c517-4f01-9136-6cc840eb369c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:50.056840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68516074-c517-4f01-9136-6cc840eb369c admin admin] PATH_INFO: `/v3/projects/a5881c187afc4c4fb48210a95bb047cf/users/1a14627b63ea49f5aa251d460b5da4bf/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:50.064169 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68516074-c517-4f01-9136-6cc840eb369c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5881c187afc4c4fb48210a95bb047cf, user_id=1a14627b63ea49f5aa251d460b5da4bf, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:50.072990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68516074-c517-4f01-9136-6cc840eb369c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:50.118203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1030/2058] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:50 2022] PUT /identity/v3/projects/a5881c187afc4c4fb48210a95bb047cf/users/1a14627b63ea49f5aa251d460b5da4bf/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:50.122215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7d313-383c-4baf-a35f-7a494dc47556 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:50.122700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7d313-383c-4baf-a35f-7a494dc47556 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:50.123201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7d313-383c-4baf-a35f-7a494dc47556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:50.185386 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-37b7d313-383c-4baf-a35f-7a494dc47556 None None] MFA Rules not processed for user `1a14627b63ea49f5aa251d460b5da4bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:50.187092 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-37b7d313-383c-4baf-a35f-7a494dc47556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:50.318079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1029/2059] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:50 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:50.329535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68516074-c517-4f01-9136-6cc840eb369c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:50.332407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-68516074-c517-4f01-9136-6cc840eb369c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:50.423187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68516074-c517-4f01-9136-6cc840eb369c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:50.430091 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a683bc1-6e9d-47c0-b25e-8492eb5672ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:50.430774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a683bc1-6e9d-47c0-b25e-8492eb5672ab service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:50.431141 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a683bc1-6e9d-47c0-b25e-8492eb5672ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:50.431476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a683bc1-6e9d-47c0-b25e-8492eb5672ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:50.444620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a683bc1-6e9d-47c0-b25e-8492eb5672ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:50.446155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a683bc1-6e9d-47c0-b25e-8492eb5672ab service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:50.472458 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1031/2060] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:50 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:52.701948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7d313-383c-4baf-a35f-7a494dc47556 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:52.727206 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:52.784927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7d313-383c-4baf-a35f-7a494dc47556 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:52.789408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:52.790058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:52.790320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:52.790622 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] PATH_INFO: `/v3/users/26084bb188d34d1e9647e4964e839e70` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:52.791853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26084bb188d34d1e9647e4964e839e70)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:52.794440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:52.846213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26084bb188d34d1e9647e4964e839e70'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:52.859275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26084bb188d34d1e9647e4964e839e70'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:52.860104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26084bb188d34d1e9647e4964e839e70'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:52.868690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26084bb188d34d1e9647e4964e839e70'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:52.878121 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26084bb188d34d1e9647e4964e839e70'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:52.886016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1030/2061] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:52 2022] DELETE /identity/v3/users/26084bb188d34d1e9647e4964e839e70 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:52.890544 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c51de-1fdd-4eec-aa90-19142d1d5749 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:52.891056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c51de-1fdd-4eec-aa90-19142d1d5749 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:52.891056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c51de-1fdd-4eec-aa90-19142d1d5749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:52.912793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-178c51de-1fdd-4eec-aa90-19142d1d5749 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:52.914038 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-178c51de-1fdd-4eec-aa90-19142d1d5749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:52.989986 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1032/2062] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:53.005093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:53.006595 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:53.095884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d057b-86fa-4e2f-be34-c371397d4665 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:53.100736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691d6a9-161f-443f-8df3-7fd6f1b156f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:53.101624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691d6a9-161f-443f-8df3-7fd6f1b156f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:53.102108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691d6a9-161f-443f-8df3-7fd6f1b156f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:53.102345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691d6a9-161f-443f-8df3-7fd6f1b156f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:53.114255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691d6a9-161f-443f-8df3-7fd6f1b156f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:53.115775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691d6a9-161f-443f-8df3-7fd6f1b156f0 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:53.132620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1031/2063] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:53.513684 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c51de-1fdd-4eec-aa90-19142d1d5749 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:53.517229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-178c51de-1fdd-4eec-aa90-19142d1d5749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:53.545742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c51de-1fdd-4eec-aa90-19142d1d5749 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:53.557080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:53.559174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:53.559473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:53.559785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] PATH_INFO: `/v3/projects/6951ba89bce243b484e35fdab09036e1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:53.561144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6951ba89bce243b484e35fdab09036e1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:53.563369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:53.628578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6951ba89bce243b484e35fdab09036e1', 'request_id': 'req-bed01b99-5d45-478a-a18f-76a0838e5b14'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:53.641746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6951ba89bce243b484e35fdab09036e1', 'request_id': 'req-bed01b99-5d45-478a-a18f-76a0838e5b14'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:53.651769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6951ba89bce243b484e35fdab09036e1', 'request_id': 'req-bed01b99-5d45-478a-a18f-76a0838e5b14'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:53.662422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:53.669770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] The token cache is being invalidate because project 6951ba89bce243b484e35fdab09036e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:53.669884 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:53.671276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:53.674531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1033/2064] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:53 2022] DELETE /identity/v3/projects/6951ba89bce243b484e35fdab09036e1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:53.678853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691d6a9-161f-443f-8df3-7fd6f1b156f0 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:53.697711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691d6a9-161f-443f-8df3-7fd6f1b156f0 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:53.759519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:53.782821 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8b4c6-9a02-451d-8e53-6ee3814bd085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:53.783484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8b4c6-9a02-451d-8e53-6ee3814bd085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:53.783771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8b4c6-9a02-451d-8e53-6ee3814bd085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:53.784054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8b4c6-9a02-451d-8e53-6ee3814bd085 admin admin] PATH_INFO: `/v3/users/1a14627b63ea49f5aa251d460b5da4bf` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:53.785269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8b4c6-9a02-451d-8e53-6ee3814bd085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a14627b63ea49f5aa251d460b5da4bf)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:53.787259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8b4c6-9a02-451d-8e53-6ee3814bd085 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:53.839508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0bb8b4c6-9a02-451d-8e53-6ee3814bd085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a14627b63ea49f5aa251d460b5da4bf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:53.851028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0bb8b4c6-9a02-451d-8e53-6ee3814bd085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a14627b63ea49f5aa251d460b5da4bf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:53.851711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0bb8b4c6-9a02-451d-8e53-6ee3814bd085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a14627b63ea49f5aa251d460b5da4bf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:53.858541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0bb8b4c6-9a02-451d-8e53-6ee3814bd085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a14627b63ea49f5aa251d460b5da4bf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:53.867796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0bb8b4c6-9a02-451d-8e53-6ee3814bd085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a14627b63ea49f5aa251d460b5da4bf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:53.874003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1032/2065] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:53 2022] DELETE /identity/v3/users/1a14627b63ea49f5aa251d460b5da4bf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:54.165835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:54.167491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:54.183092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed01b99-5d45-478a-a18f-76a0838e5b14 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:54.244610 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab6d6d-8250-420b-9639-92639c38854c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:54.245283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab6d6d-8250-420b-9639-92639c38854c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:54.245571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab6d6d-8250-420b-9639-92639c38854c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:54.245856 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab6d6d-8250-420b-9639-92639c38854c admin admin] PATH_INFO: `/v3/projects/a5881c187afc4c4fb48210a95bb047cf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:54.247025 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab6d6d-8250-420b-9639-92639c38854c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5881c187afc4c4fb48210a95bb047cf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:54.249021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab6d6d-8250-420b-9639-92639c38854c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:54.295097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-54ab6d6d-8250-420b-9639-92639c38854c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5881c187afc4c4fb48210a95bb047cf', 'request_id': 'req-54ab6d6d-8250-420b-9639-92639c38854c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:54.306105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-54ab6d6d-8250-420b-9639-92639c38854c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5881c187afc4c4fb48210a95bb047cf', 'request_id': 'req-54ab6d6d-8250-420b-9639-92639c38854c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:54.312335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-54ab6d6d-8250-420b-9639-92639c38854c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5881c187afc4c4fb48210a95bb047cf', 'request_id': 'req-54ab6d6d-8250-420b-9639-92639c38854c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:54.325015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-54ab6d6d-8250-420b-9639-92639c38854c admin admin] The token cache is being invalidate because project a5881c187afc4c4fb48210a95bb047cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:54.325311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-54ab6d6d-8250-420b-9639-92639c38854c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:54.325988 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-54ab6d6d-8250-420b-9639-92639c38854c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:54.327552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1034/2066] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:54 2022] DELETE /identity/v3/projects/a5881c187afc4c4fb48210a95bb047cf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:54.332663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1aef4-5c46-4090-a6e0-f1406023ea70 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:54.332953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1aef4-5c46-4090-a6e0-f1406023ea70 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:54.333235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1aef4-5c46-4090-a6e0-f1406023ea70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:54.354066 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2fa1aef4-5c46-4090-a6e0-f1406023ea70 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:54.355441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2fa1aef4-5c46-4090-a6e0-f1406023ea70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:54.415836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1033/2067] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:54.425221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9bd27-a079-4b74-a316-d87ccb09ae80 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:54.425602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9bd27-a079-4b74-a316-d87ccb09ae80 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:54.425799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9bd27-a079-4b74-a316-d87ccb09ae80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:54.459981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5aa9bd27-a079-4b74-a316-d87ccb09ae80 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:54.461192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5aa9bd27-a079-4b74-a316-d87ccb09ae80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:54.471878 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1035/2068] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:54.476220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1aef4-5c46-4090-a6e0-f1406023ea70 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:54.486068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1aef4-5c46-4090-a6e0-f1406023ea70 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:54.494292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be1988-d7f3-4bbd-819a-8e379ecff283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:54.495100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be1988-d7f3-4bbd-819a-8e379ecff283 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:54.495392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be1988-d7f3-4bbd-819a-8e379ecff283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:54.495903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be1988-d7f3-4bbd-819a-8e379ecff283 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:54.496616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be1988-d7f3-4bbd-819a-8e379ecff283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:54.496924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be1988-d7f3-4bbd-819a-8e379ecff283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:54.498532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be1988-d7f3-4bbd-819a-8e379ecff283 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:54.515468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1034/2069] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:38:54.516256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9bd27-a079-4b74-a316-d87ccb09ae80 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:54.535419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be1988-d7f3-4bbd-819a-8e379ecff283 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:54.539041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-53be1988-d7f3-4bbd-819a-8e379ecff283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:54.547086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9bd27-a079-4b74-a316-d87ccb09ae80 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:54.556407 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be1988-d7f3-4bbd-819a-8e379ecff283 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:54.565818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e4171-1a4f-4742-8bae-6b21759a11c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:54.566794 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e4171-1a4f-4742-8bae-6b21759a11c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:54.567147 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e4171-1a4f-4742-8bae-6b21759a11c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:54.567478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e4171-1a4f-4742-8bae-6b21759a11c5 admin admin] PATH_INFO: `/v3/users/276c3c1a8b26433298993f70984a7d8d` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:54.571254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e4171-1a4f-4742-8bae-6b21759a11c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=276c3c1a8b26433298993f70984a7d8d)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:54.573268 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388bbe-96e5-4c8b-be94-08398927e285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:54.574093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388bbe-96e5-4c8b-be94-08398927e285 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:54.574536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388bbe-96e5-4c8b-be94-08398927e285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:54.574973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56388bbe-96e5-4c8b-be94-08398927e285 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:54.575119 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e4171-1a4f-4742-8bae-6b21759a11c5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:54.575953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56388bbe-96e5-4c8b-be94-08398927e285 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:54.578023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56388bbe-96e5-4c8b-be94-08398927e285 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:54.638397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1036/2070] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:54 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:54.639422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:54.639674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388bbe-96e5-4c8b-be94-08398927e285 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:54.648478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ea2e4171-1a4f-4742-8bae-6b21759a11c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '276c3c1a8b26433298993f70984a7d8d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:54.665105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ea2e4171-1a4f-4742-8bae-6b21759a11c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '276c3c1a8b26433298993f70984a7d8d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:54.665904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ea2e4171-1a4f-4742-8bae-6b21759a11c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '276c3c1a8b26433298993f70984a7d8d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:54.677049 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ea2e4171-1a4f-4742-8bae-6b21759a11c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '276c3c1a8b26433298993f70984a7d8d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:54.687776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ea2e4171-1a4f-4742-8bae-6b21759a11c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '276c3c1a8b26433298993f70984a7d8d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:54.699168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1035/2071] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:54 2022] DELETE /identity/v3/users/276c3c1a8b26433298993f70984a7d8d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:38:54.700487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1a987-ecaa-4b00-a6bd-c3f74ee3032f None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:54.700775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1a987-ecaa-4b00-a6bd-c3f74ee3032f None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:54.701054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1a987-ecaa-4b00-a6bd-c3f74ee3032f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:54.723646 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d9f1a987-ecaa-4b00-a6bd-c3f74ee3032f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:54.724984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d9f1a987-ecaa-4b00-a6bd-c3f74ee3032f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:54.740504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56388bbe-96e5-4c8b-be94-08398927e285 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:54.747434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:54.816245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef535eab-5b12-4b50-b0f7-f41d0577365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:54.816953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef535eab-5b12-4b50-b0f7-f41d0577365e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:54.817310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef535eab-5b12-4b50-b0f7-f41d0577365e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:54.817688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef535eab-5b12-4b50-b0f7-f41d0577365e admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:54.818437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef535eab-5b12-4b50-b0f7-f41d0577365e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:54.818556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1036/2072] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:54.821102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef535eab-5b12-4b50-b0f7-f41d0577365e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:54.828456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1a987-ecaa-4b00-a6bd-c3f74ee3032f None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:54.836165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d9f1a987-ecaa-4b00-a6bd-c3f74ee3032f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:54.873810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1037/2073] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:54 2022] POST /identity/v3/users => generated 291 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:38:54.880342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef535eab-5b12-4b50-b0f7-f41d0577365e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:54.882303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ef535eab-5b12-4b50-b0f7-f41d0577365e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:54.897338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef535eab-5b12-4b50-b0f7-f41d0577365e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:54.902822 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43e660-4d57-4e34-846f-fc5ce9f69889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:54.903611 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43e660-4d57-4e34-846f-fc5ce9f69889 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:54.903902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43e660-4d57-4e34-846f-fc5ce9f69889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:54.904184 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43e660-4d57-4e34-846f-fc5ce9f69889 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:54.904881 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43e660-4d57-4e34-846f-fc5ce9f69889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:54.906472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43e660-4d57-4e34-846f-fc5ce9f69889 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:54.917856 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1038/2074] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:54.921175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43e660-4d57-4e34-846f-fc5ce9f69889 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:54.923312 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1a987-ecaa-4b00-a6bd-c3f74ee3032f None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:54.927056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f8e76-c523-4374-bb0c-e3e9b3f6f5e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:54.927712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f8e76-c523-4374-bb0c-e3e9b3f6f5e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:54.927998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f8e76-c523-4374-bb0c-e3e9b3f6f5e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:54.928281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f8e76-c523-4374-bb0c-e3e9b3f6f5e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:54.935358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43e660-4d57-4e34-846f-fc5ce9f69889 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:54.938031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f8e76-c523-4374-bb0c-e3e9b3f6f5e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:54.940118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f8e76-c523-4374-bb0c-e3e9b3f6f5e6 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:54.944216 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854696f-82f8-4609-a85c-54ffcfb18a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:54.944905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854696f-82f8-4609-a85c-54ffcfb18a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:54.945199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854696f-82f8-4609-a85c-54ffcfb18a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:54.945471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854696f-82f8-4609-a85c-54ffcfb18a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:54.946198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854696f-82f8-4609-a85c-54ffcfb18a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:54.947926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854696f-82f8-4609-a85c-54ffcfb18a31 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:54.951713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1037/2075] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:38:54.965655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1039/2076] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:38:54.967853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f8e76-c523-4374-bb0c-e3e9b3f6f5e6 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:54.977769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f8e76-c523-4374-bb0c-e3e9b3f6f5e6 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:54.983488 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6809b-c402-4626-bcef-7bbd2f9379e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:54.984181 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6809b-c402-4626-bcef-7bbd2f9379e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:54.984559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6809b-c402-4626-bcef-7bbd2f9379e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:54.984834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6809b-c402-4626-bcef-7bbd2f9379e8 admin admin] PATH_INFO: `/v3/projects/07ea20fb761e4f9ebb73b251c5d9821c/users/f94e2b8992b34cb28e29d7f5c4d545f9/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.003295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6809b-c402-4626-bcef-7bbd2f9379e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ea20fb761e4f9ebb73b251c5d9821c, user_id=f94e2b8992b34cb28e29d7f5c4d545f9, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:55.006532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6809b-c402-4626-bcef-7bbd2f9379e8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:55.021099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1038/2077] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:54 2022] PUT /identity/v3/projects/07ea20fb761e4f9ebb73b251c5d9821c/users/f94e2b8992b34cb28e29d7f5c4d545f9/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:55.034479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42279677-ade6-481d-8f2c-a06c8cb3b215 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.035508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42279677-ade6-481d-8f2c-a06c8cb3b215 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.035861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42279677-ade6-481d-8f2c-a06c8cb3b215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.068144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-42279677-ade6-481d-8f2c-a06c8cb3b215 None None] MFA Rules not processed for user `f94e2b8992b34cb28e29d7f5c4d545f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:55.069309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-42279677-ade6-481d-8f2c-a06c8cb3b215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:55.186061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1040/2078] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:55.191822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6809b-c402-4626-bcef-7bbd2f9379e8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:55.249037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:55.285237 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6809b-c402-4626-bcef-7bbd2f9379e8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:55.286959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42279677-ade6-481d-8f2c-a06c8cb3b215 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:55.288403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-42279677-ade6-481d-8f2c-a06c8cb3b215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:55.288515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf011a-bd1f-4b3b-966d-392af458037b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:55.289090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf011a-bd1f-4b3b-966d-392af458037b admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.289378 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf011a-bd1f-4b3b-966d-392af458037b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.289727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf011a-bd1f-4b3b-966d-392af458037b admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.290540 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf011a-bd1f-4b3b-966d-392af458037b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:55.292596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf011a-bd1f-4b3b-966d-392af458037b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:55.305117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42279677-ade6-481d-8f2c-a06c8cb3b215 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:55.317907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e59081-d32c-44aa-92f8-00b069b83a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:55.318747 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e59081-d32c-44aa-92f8-00b069b83a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.319017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e59081-d32c-44aa-92f8-00b069b83a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.319393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e59081-d32c-44aa-92f8-00b069b83a0c admin admin] PATH_INFO: `/v3/projects/479b2b033c1c45a3bb419342a1d16027` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.320664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e59081-d32c-44aa-92f8-00b069b83a0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=479b2b033c1c45a3bb419342a1d16027)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:55.323632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e59081-d32c-44aa-92f8-00b069b83a0c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:55.351130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1039/2079] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:55 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:55.354197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf011a-bd1f-4b3b-966d-392af458037b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:55.386292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8e59081-d32c-44aa-92f8-00b069b83a0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '479b2b033c1c45a3bb419342a1d16027', 'request_id': 'req-a8e59081-d32c-44aa-92f8-00b069b83a0c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:55.401407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8e59081-d32c-44aa-92f8-00b069b83a0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '479b2b033c1c45a3bb419342a1d16027', 'request_id': 'req-a8e59081-d32c-44aa-92f8-00b069b83a0c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:55.404055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:55.413652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8e59081-d32c-44aa-92f8-00b069b83a0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '479b2b033c1c45a3bb419342a1d16027', 'request_id': 'req-a8e59081-d32c-44aa-92f8-00b069b83a0c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:55.431261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8e59081-d32c-44aa-92f8-00b069b83a0c admin admin] The token cache is being invalidate because project 479b2b033c1c45a3bb419342a1d16027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:38:55.431610 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8e59081-d32c-44aa-92f8-00b069b83a0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:55.432241 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf011a-bd1f-4b3b-966d-392af458037b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:55.432354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a8e59081-d32c-44aa-92f8-00b069b83a0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:55.437336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1041/2080] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:38:55 2022] DELETE /identity/v3/projects/479b2b033c1c45a3bb419342a1d16027 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:38:55.439103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596b2f0-68a4-4331-a9ba-44ec2e1e1909 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.439103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596b2f0-68a4-4331-a9ba-44ec2e1e1909 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.440018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596b2f0-68a4-4331-a9ba-44ec2e1e1909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.461942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9596b2f0-68a4-4331-a9ba-44ec2e1e1909 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:55.463459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9596b2f0-68a4-4331-a9ba-44ec2e1e1909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:55.502400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be514117-f526-4b89-9d9f-3bc5103b9507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:55.503781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be514117-f526-4b89-9d9f-3bc5103b9507 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.504087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be514117-f526-4b89-9d9f-3bc5103b9507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.504353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be514117-f526-4b89-9d9f-3bc5103b9507 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.505747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be514117-f526-4b89-9d9f-3bc5103b9507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:55.508343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be514117-f526-4b89-9d9f-3bc5103b9507 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:55.536537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1040/2081] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:55 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:55.540510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be514117-f526-4b89-9d9f-3bc5103b9507 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:55.542161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-be514117-f526-4b89-9d9f-3bc5103b9507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:55.550024 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1042/2082] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:55.558019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484f007-c136-4871-8ea1-ce6af63989a9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.558555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484f007-c136-4871-8ea1-ce6af63989a9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.558993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484f007-c136-4871-8ea1-ce6af63989a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.559981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be514117-f526-4b89-9d9f-3bc5103b9507 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:55.568583 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690888ec-973b-4e54-bce9-4dfd581ce410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:55.569481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690888ec-973b-4e54-bce9-4dfd581ce410 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.569743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690888ec-973b-4e54-bce9-4dfd581ce410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.570100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690888ec-973b-4e54-bce9-4dfd581ce410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.571019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690888ec-973b-4e54-bce9-4dfd581ce410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:55.572802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690888ec-973b-4e54-bce9-4dfd581ce410 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:55.587295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a484f007-c136-4871-8ea1-ce6af63989a9 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:55.588559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a484f007-c136-4871-8ea1-ce6af63989a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:55.589962 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1041/2083] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:38:55.594280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690888ec-973b-4e54-bce9-4dfd581ce410 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:55.603002 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1043/2084] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:55.606998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484f007-c136-4871-8ea1-ce6af63989a9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:55.612986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690888ec-973b-4e54-bce9-4dfd581ce410 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:55.617523 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484f007-c136-4871-8ea1-ce6af63989a9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:55.618369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199ac2a-0363-4f4e-97ae-9df688a8232b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:55.619041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199ac2a-0363-4f4e-97ae-9df688a8232b admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.619471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199ac2a-0363-4f4e-97ae-9df688a8232b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.619775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199ac2a-0363-4f4e-97ae-9df688a8232b admin admin] PATH_INFO: `/v3/projects/2807c5a4aa57460faf4746136f320130/users/412dbd1c78c04f2b976ce4ea03ddfd41/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.622748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642e0e2-93eb-4c8f-9ece-7e79ddbf4ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:55.623490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642e0e2-93eb-4c8f-9ece-7e79ddbf4ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.623777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642e0e2-93eb-4c8f-9ece-7e79ddbf4ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.624127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642e0e2-93eb-4c8f-9ece-7e79ddbf4ea1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.624824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642e0e2-93eb-4c8f-9ece-7e79ddbf4ea1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:38:55.625127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642e0e2-93eb-4c8f-9ece-7e79ddbf4ea1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:55.626815 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642e0e2-93eb-4c8f-9ece-7e79ddbf4ea1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:55.636564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199ac2a-0363-4f4e-97ae-9df688a8232b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2807c5a4aa57460faf4746136f320130, user_id=412dbd1c78c04f2b976ce4ea03ddfd41, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:55.638922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1044/2085] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:38:55.639281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199ac2a-0363-4f4e-97ae-9df688a8232b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:55.643218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642e0e2-93eb-4c8f-9ece-7e79ddbf4ea1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:55.654257 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642e0e2-93eb-4c8f-9ece-7e79ddbf4ea1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:55.655115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1042/2086] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:55 2022] PUT /identity/v3/projects/2807c5a4aa57460faf4746136f320130/users/412dbd1c78c04f2b976ce4ea03ddfd41/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:38:55.662244 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199ac2a-0363-4f4e-97ae-9df688a8232b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:55.706803 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:55.736059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd0717-c90c-4ffe-b951-66a839e1322e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:55.737370 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd0717-c90c-4ffe-b951-66a839e1322e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.738251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd0717-c90c-4ffe-b951-66a839e1322e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.739236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd0717-c90c-4ffe-b951-66a839e1322e admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.740410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd0717-c90c-4ffe-b951-66a839e1322e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:55.743809 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd0717-c90c-4ffe-b951-66a839e1322e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:55.754746 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199ac2a-0363-4f4e-97ae-9df688a8232b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:55.761283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1ee5b-b8fc-49b7-9821-526176d6345d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:55.761283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1ee5b-b8fc-49b7-9821-526176d6345d admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.761594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1ee5b-b8fc-49b7-9821-526176d6345d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.761669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1ee5b-b8fc-49b7-9821-526176d6345d admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.762327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1ee5b-b8fc-49b7-9821-526176d6345d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:55.763998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1ee5b-b8fc-49b7-9821-526176d6345d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:55.779623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1043/2087] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:55.788666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1ee5b-b8fc-49b7-9821-526176d6345d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:55.794342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1045/2088] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:38:55 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:38:55.797392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1ee5b-b8fc-49b7-9821-526176d6345d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:55.798036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd0717-c90c-4ffe-b951-66a839e1322e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:55.876285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:55.900486 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd0717-c90c-4ffe-b951-66a839e1322e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:55.909273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d0aae-cb11-4570-9c9e-2d7f65c3a646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:55.909273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d0aae-cb11-4570-9c9e-2d7f65c3a646 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.909273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d0aae-cb11-4570-9c9e-2d7f65c3a646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.910035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d0aae-cb11-4570-9c9e-2d7f65c3a646 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.910834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d0aae-cb11-4570-9c9e-2d7f65c3a646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:55.914187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d0aae-cb11-4570-9c9e-2d7f65c3a646 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:55.918668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767e3e1-e73b-42d9-bb12-bc068e04dcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:55.919861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767e3e1-e73b-42d9-bb12-bc068e04dcb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.920516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767e3e1-e73b-42d9-bb12-bc068e04dcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.923160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767e3e1-e73b-42d9-bb12-bc068e04dcb6 admin admin] PATH_INFO: `/v3/projects/2807c5a4aa57460faf4746136f320130/users/412dbd1c78c04f2b976ce4ea03ddfd41/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.930429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767e3e1-e73b-42d9-bb12-bc068e04dcb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2807c5a4aa57460faf4746136f320130, user_id=412dbd1c78c04f2b976ce4ea03ddfd41, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:55.933364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767e3e1-e73b-42d9-bb12-bc068e04dcb6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:55.963282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1046/2089] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:38:55 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:38:55.967670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d0aae-cb11-4570-9c9e-2d7f65c3a646 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:55.974178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1044/2090] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:55 2022] PUT /identity/v3/projects/2807c5a4aa57460faf4746136f320130/users/412dbd1c78c04f2b976ce4ea03ddfd41/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:55.990888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a67d22-8d04-426c-90fb-0f58359dd16c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:55.991231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a67d22-8d04-426c-90fb-0f58359dd16c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:55.991515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a67d22-8d04-426c-90fb-0f58359dd16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:55.997183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d0aae-cb11-4570-9c9e-2d7f65c3a646 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:56.025527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-15a67d22-8d04-426c-90fb-0f58359dd16c None None] MFA Rules not processed for user `412dbd1c78c04f2b976ce4ea03ddfd41`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:56.026763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-15a67d22-8d04-426c-90fb-0f58359dd16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:56.098187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43afed-a3af-4469-951d-797fcbbe8cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:56.098824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43afed-a3af-4469-951d-797fcbbe8cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:56.099158 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43afed-a3af-4469-951d-797fcbbe8cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:56.099584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43afed-a3af-4469-951d-797fcbbe8cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:56.100304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43afed-a3af-4469-951d-797fcbbe8cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:56.102256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43afed-a3af-4469-951d-797fcbbe8cc8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:56.119408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1047/2091] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:38:56.120945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43afed-a3af-4469-951d-797fcbbe8cc8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:56.136223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43afed-a3af-4469-951d-797fcbbe8cc8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:56.142453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d556690-21fe-4eaf-8b74-c951ee5df45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:56.143597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d556690-21fe-4eaf-8b74-c951ee5df45e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:56.146430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d556690-21fe-4eaf-8b74-c951ee5df45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:56.146751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d556690-21fe-4eaf-8b74-c951ee5df45e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:56.148290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d556690-21fe-4eaf-8b74-c951ee5df45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:56.150324 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d556690-21fe-4eaf-8b74-c951ee5df45e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:56.168555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1045/2092] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:56.181423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1048/2093] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:38:56.184751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a67d22-8d04-426c-90fb-0f58359dd16c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:56.185987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d556690-21fe-4eaf-8b74-c951ee5df45e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:56.188942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-15a67d22-8d04-426c-90fb-0f58359dd16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:56.197131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d556690-21fe-4eaf-8b74-c951ee5df45e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:56.201901 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3793e-742d-460a-add5-cfda50492b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:56.202570 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3793e-742d-460a-add5-cfda50492b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:56.202858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3793e-742d-460a-add5-cfda50492b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:56.203178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3793e-742d-460a-add5-cfda50492b37 admin admin] PATH_INFO: `/v3/projects/08a5b46044a743dcb9a9a341c241f94f/users/5aff46d914a94ff38057bfbc0015cdfc/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:56.221424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3793e-742d-460a-add5-cfda50492b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a5b46044a743dcb9a9a341c241f94f, user_id=5aff46d914a94ff38057bfbc0015cdfc, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:56.224647 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3793e-742d-460a-add5-cfda50492b37 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:56.249211 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:56.250808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1049/2094] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:38:56 2022] PUT /identity/v3/projects/08a5b46044a743dcb9a9a341c241f94f/users/5aff46d914a94ff38057bfbc0015cdfc/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:56.262770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b56e2-627f-490f-b231-cb07d07658ad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:56.263255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b56e2-627f-490f-b231-cb07d07658ad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:56.263366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b56e2-627f-490f-b231-cb07d07658ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:56.276977 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a67d22-8d04-426c-90fb-0f58359dd16c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:56.300627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f26b56e2-627f-490f-b231-cb07d07658ad None None] MFA Rules not processed for user `5aff46d914a94ff38057bfbc0015cdfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:56.303804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f26b56e2-627f-490f-b231-cb07d07658ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:56.341414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb893028-2e46-491a-b2f3-08f02bd2a937 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:56.342057 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb893028-2e46-491a-b2f3-08f02bd2a937 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:56.343961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb893028-2e46-491a-b2f3-08f02bd2a937 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:56.344378 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb893028-2e46-491a-b2f3-08f02bd2a937 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:56.396533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1050/2095] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:38:56.431053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb893028-2e46-491a-b2f3-08f02bd2a937 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:56.433475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb893028-2e46-491a-b2f3-08f02bd2a937 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:56.447703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1046/2096] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:56.474719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b56e2-627f-490f-b231-cb07d07658ad None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:56.484907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b56e2-627f-490f-b231-cb07d07658ad None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:56.489481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809142e-164b-47dc-80e0-dff2750f4bca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:56.489946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809142e-164b-47dc-80e0-dff2750f4bca service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:56.490228 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809142e-164b-47dc-80e0-dff2750f4bca service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:56.490504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809142e-164b-47dc-80e0-dff2750f4bca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:56.499305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809142e-164b-47dc-80e0-dff2750f4bca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:56.500888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809142e-164b-47dc-80e0-dff2750f4bca service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:56.515050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1051/2097] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:38:56.692902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb893028-2e46-491a-b2f3-08f02bd2a937 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:56.694319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-bb893028-2e46-491a-b2f3-08f02bd2a937 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:56.709500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb893028-2e46-491a-b2f3-08f02bd2a937 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:56.714167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d443bc-ece3-4891-92af-88986a6edf24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:56.714822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d443bc-ece3-4891-92af-88986a6edf24 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:56.715577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d443bc-ece3-4891-92af-88986a6edf24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:56.715906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d443bc-ece3-4891-92af-88986a6edf24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:56.717954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c8d443bc-ece3-4891-92af-88986a6edf24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:56.787297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d443bc-ece3-4891-92af-88986a6edf24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:56.788756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d443bc-ece3-4891-92af-88986a6edf24 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:56.800798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1047/2098] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:59.478608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809142e-164b-47dc-80e0-dff2750f4bca service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:59.499426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:59.546306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809142e-164b-47dc-80e0-dff2750f4bca service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:59.551097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:59.551730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:59.552012 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:59.552284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] PATH_INFO: `/v3/users/f94e2b8992b34cb28e29d7f5c4d545f9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:59.553588 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f94e2b8992b34cb28e29d7f5c4d545f9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:59.555831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:59.606685 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f94e2b8992b34cb28e29d7f5c4d545f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:59.617258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f94e2b8992b34cb28e29d7f5c4d545f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:59.634338 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f94e2b8992b34cb28e29d7f5c4d545f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:59.635110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f94e2b8992b34cb28e29d7f5c4d545f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:59.635898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f94e2b8992b34cb28e29d7f5c4d545f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:59.649310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1052/2099] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:59 2022] DELETE /identity/v3/users/f94e2b8992b34cb28e29d7f5c4d545f9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:59.653811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beba641-3603-4f78-a4c3-573b2cf0b451 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:59.654255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beba641-3603-4f78-a4c3-573b2cf0b451 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:59.654402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beba641-3603-4f78-a4c3-573b2cf0b451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:59.675970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0beba641-3603-4f78-a4c3-573b2cf0b451 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:59.677255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0beba641-3603-4f78-a4c3-573b2cf0b451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:59.722300 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:59.745413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1048/2100] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:38:59.758905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:59.760388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:59.781006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beba641-3603-4f78-a4c3-573b2cf0b451 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:38:59.782958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0beba641-3603-4f78-a4c3-573b2cf0b451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:59.796233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beba641-3603-4f78-a4c3-573b2cf0b451 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:59.801532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:59.802181 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:59.802463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:59.802757 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] PATH_INFO: `/v3/users/5aff46d914a94ff38057bfbc0015cdfc` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:59.803929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aff46d914a94ff38057bfbc0015cdfc)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:59.805855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:59.821822 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439250c-c355-4972-9ea7-6530cb9dd31b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:38:59.826133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd07d6b-128f-4e89-b194-cde965cedfe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:38:59.826854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd07d6b-128f-4e89-b194-cde965cedfe7 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:59.827190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd07d6b-128f-4e89-b194-cde965cedfe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:59.827477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd07d6b-128f-4e89-b194-cde965cedfe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:59.839957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd07d6b-128f-4e89-b194-cde965cedfe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:38:59.841577 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd07d6b-128f-4e89-b194-cde965cedfe7 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:38:59.854268 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1053/2101] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:38:59.857717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aff46d914a94ff38057bfbc0015cdfc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:59.871971 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aff46d914a94ff38057bfbc0015cdfc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:59.872704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aff46d914a94ff38057bfbc0015cdfc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:59.881349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aff46d914a94ff38057bfbc0015cdfc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:59.889280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aff46d914a94ff38057bfbc0015cdfc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:38:59.897715 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1049/2102] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:38:59 2022] DELETE /identity/v3/users/5aff46d914a94ff38057bfbc0015cdfc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:38:59.902753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1d54e-0ee9-4871-a8c7-da1ea1e91a05 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:38:59.903561 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1d54e-0ee9-4871-a8c7-da1ea1e91a05 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:38:59.903842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1d54e-0ee9-4871-a8c7-da1ea1e91a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:38:59.924973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-92d1d54e-0ee9-4871-a8c7-da1ea1e91a05 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:38:59.925847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-92d1d54e-0ee9-4871-a8c7-da1ea1e91a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:38:59.971579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:38:59.996004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1054/2103] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:00.009033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:00.010625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:00.092825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7612a90-a905-4e9e-9855-b60913ee2a37 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:00.097573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9318fd-395c-41e1-9125-0ca3132c435a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:00.098303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9318fd-395c-41e1-9125-0ca3132c435a service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:00.098588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9318fd-395c-41e1-9125-0ca3132c435a service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:00.098859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9318fd-395c-41e1-9125-0ca3132c435a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:00.108943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9318fd-395c-41e1-9125-0ca3132c435a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:00.110504 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9318fd-395c-41e1-9125-0ca3132c435a service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:00.126598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1050/2104] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:00.187574 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1d54e-0ee9-4871-a8c7-da1ea1e91a05 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:00.189581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-92d1d54e-0ee9-4871-a8c7-da1ea1e91a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:00.218972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1d54e-0ee9-4871-a8c7-da1ea1e91a05 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:00.240236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:00.240796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:00.241111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:00.241855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] PATH_INFO: `/v3/projects/07ea20fb761e4f9ebb73b251c5d9821c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:00.243109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07ea20fb761e4f9ebb73b251c5d9821c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:00.246125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:00.300864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07ea20fb761e4f9ebb73b251c5d9821c', 'request_id': 'req-df158aac-d133-4a59-ba04-3685a4983c69'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.313090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07ea20fb761e4f9ebb73b251c5d9821c', 'request_id': 'req-df158aac-d133-4a59-ba04-3685a4983c69'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.323481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07ea20fb761e4f9ebb73b251c5d9821c', 'request_id': 'req-df158aac-d133-4a59-ba04-3685a4983c69'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.339255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] The token cache is being invalidate because project 07ea20fb761e4f9ebb73b251c5d9821c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:00.339597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.340310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.341995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1055/2105] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:00 2022] DELETE /identity/v3/projects/07ea20fb761e4f9ebb73b251c5d9821c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:00.347521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9318fd-395c-41e1-9125-0ca3132c435a service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:00.359195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9318fd-395c-41e1-9125-0ca3132c435a service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:00.435679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb9956-14e8-4c03-b392-dddac7650590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:00.436331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb9956-14e8-4c03-b392-dddac7650590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:00.436617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb9956-14e8-4c03-b392-dddac7650590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:00.436905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb9956-14e8-4c03-b392-dddac7650590 admin admin] PATH_INFO: `/v3/users/412dbd1c78c04f2b976ce4ea03ddfd41` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:00.438087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb9956-14e8-4c03-b392-dddac7650590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=412dbd1c78c04f2b976ce4ea03ddfd41)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:00.440272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb9956-14e8-4c03-b392-dddac7650590 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:00.442301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:00.482626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:00.487205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:00.502262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df158aac-d133-4a59-ba04-3685a4983c69 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:00.518369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090178f-ed13-483f-9585-f2dc8102c11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:00.519031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090178f-ed13-483f-9585-f2dc8102c11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:00.519354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090178f-ed13-483f-9585-f2dc8102c11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:00.519638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090178f-ed13-483f-9585-f2dc8102c11d admin admin] PATH_INFO: `/v3/projects/08a5b46044a743dcb9a9a341c241f94f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:00.522793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090178f-ed13-483f-9585-f2dc8102c11d admin admin] RBAC: Authorizing `identity:delete_project(project_id=08a5b46044a743dcb9a9a341c241f94f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:00.525804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090178f-ed13-483f-9585-f2dc8102c11d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:00.528835 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33cb9956-14e8-4c03-b392-dddac7650590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '412dbd1c78c04f2b976ce4ea03ddfd41'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.549938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33cb9956-14e8-4c03-b392-dddac7650590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '412dbd1c78c04f2b976ce4ea03ddfd41'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.550712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33cb9956-14e8-4c03-b392-dddac7650590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '412dbd1c78c04f2b976ce4ea03ddfd41'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.560526 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33cb9956-14e8-4c03-b392-dddac7650590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '412dbd1c78c04f2b976ce4ea03ddfd41'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.569445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33cb9956-14e8-4c03-b392-dddac7650590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '412dbd1c78c04f2b976ce4ea03ddfd41'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.576477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1051/2106] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:00 2022] DELETE /identity/v3/users/412dbd1c78c04f2b976ce4ea03ddfd41 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:00.604663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f090178f-ed13-483f-9585-f2dc8102c11d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08a5b46044a743dcb9a9a341c241f94f', 'request_id': 'req-f090178f-ed13-483f-9585-f2dc8102c11d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.615373 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f090178f-ed13-483f-9585-f2dc8102c11d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08a5b46044a743dcb9a9a341c241f94f', 'request_id': 'req-f090178f-ed13-483f-9585-f2dc8102c11d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.623676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f090178f-ed13-483f-9585-f2dc8102c11d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08a5b46044a743dcb9a9a341c241f94f', 'request_id': 'req-f090178f-ed13-483f-9585-f2dc8102c11d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.637174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f090178f-ed13-483f-9585-f2dc8102c11d admin admin] The token cache is being invalidate because project 08a5b46044a743dcb9a9a341c241f94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:00.637394 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f090178f-ed13-483f-9585-f2dc8102c11d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.638914 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f090178f-ed13-483f-9585-f2dc8102c11d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:00.639950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1056/2107] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:00 2022] DELETE /identity/v3/projects/08a5b46044a743dcb9a9a341c241f94f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:00.645616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ce7e2-f1aa-4461-b055-77f50db5939c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:00.646257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ce7e2-f1aa-4461-b055-77f50db5939c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:00.646565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ce7e2-f1aa-4461-b055-77f50db5939c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:00.671089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ba1ce7e2-f1aa-4461-b055-77f50db5939c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:00.672635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ba1ce7e2-f1aa-4461-b055-77f50db5939c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:00.758111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1052/2108] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:39:00.766009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5a55f-0727-4461-9b4c-b1fad0372fb4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:00.766311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5a55f-0727-4461-9b4c-b1fad0372fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:00.766598 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5a55f-0727-4461-9b4c-b1fad0372fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:00.792959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-64b5a55f-0727-4461-9b4c-b1fad0372fb4 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:00.795008 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-64b5a55f-0727-4461-9b4c-b1fad0372fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:00.808235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1057/2109] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:00.809539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ce7e2-f1aa-4461-b055-77f50db5939c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:00.822602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ce7e2-f1aa-4461-b055-77f50db5939c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:00.827682 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2b55b-de50-48a1-be07-5e63075f54ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:00.828303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2b55b-de50-48a1-be07-5e63075f54ed admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:00.828584 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2b55b-de50-48a1-be07-5e63075f54ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:00.830090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2b55b-de50-48a1-be07-5e63075f54ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:00.830228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2b55b-de50-48a1-be07-5e63075f54ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:39:00.830490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2b55b-de50-48a1-be07-5e63075f54ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:00.832888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2b55b-de50-48a1-be07-5e63075f54ed admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:00.850426 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1053/2110] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:39:00.851606 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5a55f-0727-4461-9b4c-b1fad0372fb4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:00.860974 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5a55f-0727-4461-9b4c-b1fad0372fb4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:00.865888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848782d-9f3c-463d-aaa7-928d6dd20d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:00.866442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848782d-9f3c-463d-aaa7-928d6dd20d78 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:00.866724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848782d-9f3c-463d-aaa7-928d6dd20d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:00.867001 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848782d-9f3c-463d-aaa7-928d6dd20d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:00.868079 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848782d-9f3c-463d-aaa7-928d6dd20d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:00.869610 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848782d-9f3c-463d-aaa7-928d6dd20d78 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:00.909537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1058/2111] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:00 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:00.911206 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2b55b-de50-48a1-be07-5e63075f54ed admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:00.924038 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848782d-9f3c-463d-aaa7-928d6dd20d78 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:00.926102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9848782d-9f3c-463d-aaa7-928d6dd20d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:00.931323 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2b55b-de50-48a1-be07-5e63075f54ed admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:00.977628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:01.020652 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee55db-d84d-411f-83f3-073ccb471ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.021283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee55db-d84d-411f-83f3-073ccb471ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.021571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee55db-d84d-411f-83f3-073ccb471ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.021853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee55db-d84d-411f-83f3-073ccb471ef3 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.022566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee55db-d84d-411f-83f3-073ccb471ef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.024578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee55db-d84d-411f-83f3-073ccb471ef3 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.036206 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:01.037098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848782d-9f3c-463d-aaa7-928d6dd20d78 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.043427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf737783-6a5e-41be-ab72-f7d1317d1b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.044066 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf737783-6a5e-41be-ab72-f7d1317d1b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.044357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf737783-6a5e-41be-ab72-f7d1317d1b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.044669 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf737783-6a5e-41be-ab72-f7d1317d1b0a admin admin] PATH_INFO: `/v3/projects/2807c5a4aa57460faf4746136f320130` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.046047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf737783-6a5e-41be-ab72-f7d1317d1b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2807c5a4aa57460faf4746136f320130)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.048020 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf737783-6a5e-41be-ab72-f7d1317d1b0a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.052960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1054/2112] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:00 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:01.053707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee55db-d84d-411f-83f3-073ccb471ef3 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.064454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee55db-d84d-411f-83f3-073ccb471ef3 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.070138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d470b-7316-4b9d-b518-1fd65ebcaaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.070746 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d470b-7316-4b9d-b518-1fd65ebcaaf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.071360 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d470b-7316-4b9d-b518-1fd65ebcaaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.071666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d470b-7316-4b9d-b518-1fd65ebcaaf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.072528 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d470b-7316-4b9d-b518-1fd65ebcaaf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.074196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d470b-7316-4b9d-b518-1fd65ebcaaf5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.091043 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1055/2113] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:01.095948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d470b-7316-4b9d-b518-1fd65ebcaaf5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.105173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf737783-6a5e-41be-ab72-f7d1317d1b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2807c5a4aa57460faf4746136f320130', 'request_id': 'req-cf737783-6a5e-41be-ab72-f7d1317d1b0a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:01.108452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d470b-7316-4b9d-b518-1fd65ebcaaf5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.113612 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf737783-6a5e-41be-ab72-f7d1317d1b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2807c5a4aa57460faf4746136f320130', 'request_id': 'req-cf737783-6a5e-41be-ab72-f7d1317d1b0a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:01.120087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf737783-6a5e-41be-ab72-f7d1317d1b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2807c5a4aa57460faf4746136f320130', 'request_id': 'req-cf737783-6a5e-41be-ab72-f7d1317d1b0a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:01.132989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf737783-6a5e-41be-ab72-f7d1317d1b0a admin admin] The token cache is being invalidate because project 2807c5a4aa57460faf4746136f320130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:01.133452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf737783-6a5e-41be-ab72-f7d1317d1b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:01.134236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf737783-6a5e-41be-ab72-f7d1317d1b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:01.135982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1059/2114] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:00 2022] DELETE /identity/v3/projects/2807c5a4aa57460faf4746136f320130 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:01.141033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dbcb4-37da-4e23-9473-417118251545 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.141430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dbcb4-37da-4e23-9473-417118251545 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.141906 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dbcb4-37da-4e23-9473-417118251545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.167200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f59dbcb4-37da-4e23-9473-417118251545 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:01.168537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f59dbcb4-37da-4e23-9473-417118251545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:01.181670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9884b23-636e-446e-879a-33339de8309d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.182404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9884b23-636e-446e-879a-33339de8309d admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.182767 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9884b23-636e-446e-879a-33339de8309d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.183312 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9884b23-636e-446e-879a-33339de8309d admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.184471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9884b23-636e-446e-879a-33339de8309d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.186197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9884b23-636e-446e-879a-33339de8309d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.198942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1056/2115] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:01.203254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9884b23-636e-446e-879a-33339de8309d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.205894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a9884b23-636e-446e-879a-33339de8309d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:01.222029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9884b23-636e-446e-879a-33339de8309d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.236705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee47e63-d988-4274-9684-e9fc5d5e6c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.237715 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee47e63-d988-4274-9684-e9fc5d5e6c64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.238119 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee47e63-d988-4274-9684-e9fc5d5e6c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.238887 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee47e63-d988-4274-9684-e9fc5d5e6c64 admin admin] PATH_INFO: `/v3/projects/41f2241bc21f4a20a00a8a7e4b52e36c/users/e2202cff1cdb4005ab333557aa2d06bb/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.245931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1060/2116] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:01.248217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5bea6-75ed-4b20-a992-d20dbc18573d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.248445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5bea6-75ed-4b20-a992-d20dbc18573d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.248727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5bea6-75ed-4b20-a992-d20dbc18573d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.265673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee47e63-d988-4274-9684-e9fc5d5e6c64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f2241bc21f4a20a00a8a7e4b52e36c, user_id=e2202cff1cdb4005ab333557aa2d06bb, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.269894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee47e63-d988-4274-9684-e9fc5d5e6c64 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.274389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-70a5bea6-75ed-4b20-a992-d20dbc18573d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:01.276241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-70a5bea6-75ed-4b20-a992-d20dbc18573d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:01.285114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1061/2117] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:39:01.290046 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5bea6-75ed-4b20-a992-d20dbc18573d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.307117 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1057/2118] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:01 2022] PUT /identity/v3/projects/41f2241bc21f4a20a00a8a7e4b52e36c/users/e2202cff1cdb4005ab333557aa2d06bb/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:39:01.307117 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723efac7-8e50-4f1e-bb4b-d80cf08f09d8 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.307117 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723efac7-8e50-4f1e-bb4b-d80cf08f09d8 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.307117 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723efac7-8e50-4f1e-bb4b-d80cf08f09d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.310354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5bea6-75ed-4b20-a992-d20dbc18573d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.343621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-723efac7-8e50-4f1e-bb4b-d80cf08f09d8 None None] MFA Rules not processed for user `e2202cff1cdb4005ab333557aa2d06bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:01.345636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-723efac7-8e50-4f1e-bb4b-d80cf08f09d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:01.361631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:01.378988 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3009e-96e2-4179-967e-ce2310a27405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.379835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3009e-96e2-4179-967e-ce2310a27405 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.380123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3009e-96e2-4179-967e-ce2310a27405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.380402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3009e-96e2-4179-967e-ce2310a27405 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.381098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3009e-96e2-4179-967e-ce2310a27405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:39:01.381410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3009e-96e2-4179-967e-ce2310a27405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.383054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3009e-96e2-4179-967e-ce2310a27405 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.399264 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1062/2119] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:39:01.400791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3009e-96e2-4179-967e-ce2310a27405 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.411935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3009e-96e2-4179-967e-ce2310a27405 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.417813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a2896-e3f2-40bf-a68d-7bba0cb265c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.418409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a2896-e3f2-40bf-a68d-7bba0cb265c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.418916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a2896-e3f2-40bf-a68d-7bba0cb265c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.419030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a2896-e3f2-40bf-a68d-7bba0cb265c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.419714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a2896-e3f2-40bf-a68d-7bba0cb265c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.421661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a2896-e3f2-40bf-a68d-7bba0cb265c0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.422281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1058/2120] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:01.426982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723efac7-8e50-4f1e-bb4b-d80cf08f09d8 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.437202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723efac7-8e50-4f1e-bb4b-d80cf08f09d8 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.442249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e49e6-fb57-48db-be49-27ed291fcbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.443424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e49e6-fb57-48db-be49-27ed291fcbd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.443866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e49e6-fb57-48db-be49-27ed291fcbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.444155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e49e6-fb57-48db-be49-27ed291fcbd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.444845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e49e6-fb57-48db-be49-27ed291fcbd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.446908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e49e6-fb57-48db-be49-27ed291fcbd7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.453742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1063/2121] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:01 2022] POST /identity/v3/projects => generated 409 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:01.455433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a2896-e3f2-40bf-a68d-7bba0cb265c0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.481306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1059/2122] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:01 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:01.487601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e49e6-fb57-48db-be49-27ed291fcbd7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.518053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:01.549498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a2896-e3f2-40bf-a68d-7bba0cb265c0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.560258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e49e6-fb57-48db-be49-27ed291fcbd7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.564984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d89d8-6850-42c7-8b47-7a1e06c8e164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.565617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d89d8-6850-42c7-8b47-7a1e06c8e164 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.566039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d89d8-6850-42c7-8b47-7a1e06c8e164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.566329 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d89d8-6850-42c7-8b47-7a1e06c8e164 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.567255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d89d8-6850-42c7-8b47-7a1e06c8e164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.569199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d89d8-6850-42c7-8b47-7a1e06c8e164 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.609024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1060/2123] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:01 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:01.609925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d89d8-6850-42c7-8b47-7a1e06c8e164 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.621800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d89d8-6850-42c7-8b47-7a1e06c8e164 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.627939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcae56-4bf2-4083-8e4c-50c7b810706f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.628641 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcae56-4bf2-4083-8e4c-50c7b810706f admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.629160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcae56-4bf2-4083-8e4c-50c7b810706f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.629709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcae56-4bf2-4083-8e4c-50c7b810706f admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.630481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcae56-4bf2-4083-8e4c-50c7b810706f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.632199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcae56-4bf2-4083-8e4c-50c7b810706f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.635549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a951c-de58-4a6f-8bd2-e0e56b9a2242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.636274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a951c-de58-4a6f-8bd2-e0e56b9a2242 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.636687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a951c-de58-4a6f-8bd2-e0e56b9a2242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.637052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a951c-de58-4a6f-8bd2-e0e56b9a2242 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.637794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a951c-de58-4a6f-8bd2-e0e56b9a2242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.639837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a951c-de58-4a6f-8bd2-e0e56b9a2242 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.646249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1061/2124] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:01.652291 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcae56-4bf2-4083-8e4c-50c7b810706f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.665184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcae56-4bf2-4083-8e4c-50c7b810706f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.670873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08273fd-9927-42e5-b3d3-14628ea1c5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.671526 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08273fd-9927-42e5-b3d3-14628ea1c5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.672084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08273fd-9927-42e5-b3d3-14628ea1c5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.672084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08273fd-9927-42e5-b3d3-14628ea1c5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.672861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08273fd-9927-42e5-b3d3-14628ea1c5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.673872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1064/2125] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:01 2022] POST /identity/v3/users => generated 305 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:01.674833 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08273fd-9927-42e5-b3d3-14628ea1c5b4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.678298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a951c-de58-4a6f-8bd2-e0e56b9a2242 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.689739 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1062/2126] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:39:01.694072 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08273fd-9927-42e5-b3d3-14628ea1c5b4 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.705876 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08273fd-9927-42e5-b3d3-14628ea1c5b4 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.708187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a951c-de58-4a6f-8bd2-e0e56b9a2242 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.711436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e18b6-5ab1-4759-ad30-c8ca740955da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.713622 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e18b6-5ab1-4759-ad30-c8ca740955da admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.713917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e18b6-5ab1-4759-ad30-c8ca740955da admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.714039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4e443-3f43-440b-aa49-bbd74cede8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.714230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e18b6-5ab1-4759-ad30-c8ca740955da admin admin] PATH_INFO: `/v3/projects/4202efd1004b421cb58de3f8a9093425/users/3455fb5f83b541fab52dd8078684791f/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.714936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4e443-3f43-440b-aa49-bbd74cede8bc admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.715480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4e443-3f43-440b-aa49-bbd74cede8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.715763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4e443-3f43-440b-aa49-bbd74cede8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.716463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4e443-3f43-440b-aa49-bbd74cede8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.718510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4e443-3f43-440b-aa49-bbd74cede8bc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.738862 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e18b6-5ab1-4759-ad30-c8ca740955da admin admin] RBAC: Authorizing `identity:create_grant(project_id=4202efd1004b421cb58de3f8a9093425, user_id=3455fb5f83b541fab52dd8078684791f, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.743788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1065/2127] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:39:01.744034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e18b6-5ab1-4759-ad30-c8ca740955da admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.747647 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4e443-3f43-440b-aa49-bbd74cede8bc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.760994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4e443-3f43-440b-aa49-bbd74cede8bc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.762556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1063/2128] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:01 2022] PUT /identity/v3/projects/4202efd1004b421cb58de3f8a9093425/users/3455fb5f83b541fab52dd8078684791f/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:01.774866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6d95d-dafa-425b-ac15-e1baaa329692 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.775187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6d95d-dafa-425b-ac15-e1baaa329692 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.775473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6d95d-dafa-425b-ac15-e1baaa329692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.809590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-97e6d95d-dafa-425b-ac15-e1baaa329692 None None] MFA Rules not processed for user `3455fb5f83b541fab52dd8078684791f`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:01.810829 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-97e6d95d-dafa-425b-ac15-e1baaa329692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:01.838897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:01.868358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441ca3b-075a-4b1c-bad1-14700c4d3982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.869276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441ca3b-075a-4b1c-bad1-14700c4d3982 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.869559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441ca3b-075a-4b1c-bad1-14700c4d3982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.869837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441ca3b-075a-4b1c-bad1-14700c4d3982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.870607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441ca3b-075a-4b1c-bad1-14700c4d3982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.872488 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441ca3b-075a-4b1c-bad1-14700c4d3982 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.887134 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1066/2129] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:01.890734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441ca3b-075a-4b1c-bad1-14700c4d3982 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.909867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441ca3b-075a-4b1c-bad1-14700c4d3982 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.920809 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1064/2130] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:01.926070 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e8f41-fcfd-493b-982d-e4fc57cc7061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.927226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e8f41-fcfd-493b-982d-e4fc57cc7061 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.927516 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e8f41-fcfd-493b-982d-e4fc57cc7061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.927787 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e8f41-fcfd-493b-982d-e4fc57cc7061 admin admin] PATH_INFO: `/v3/projects/4c06aa6e88634a97901bd41cf05c3bb2/users/2a1901d751fe459896a55219b1f2646f/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.928453 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6d95d-dafa-425b-ac15-e1baaa329692 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:01.939788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6d95d-dafa-425b-ac15-e1baaa329692 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:01.944634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e55db-4a5e-4cf5-b7da-74996dda8802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:01.944935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e8f41-fcfd-493b-982d-e4fc57cc7061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c06aa6e88634a97901bd41cf05c3bb2, user_id=2a1901d751fe459896a55219b1f2646f, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.945340 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e55db-4a5e-4cf5-b7da-74996dda8802 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.945626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e55db-4a5e-4cf5-b7da-74996dda8802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.945900 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e55db-4a5e-4cf5-b7da-74996dda8802 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.946567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e55db-4a5e-4cf5-b7da-74996dda8802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:01.948493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e55db-4a5e-4cf5-b7da-74996dda8802 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.949287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e8f41-fcfd-493b-982d-e4fc57cc7061 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:01.967875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1067/2131] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:01 2022] PUT /identity/v3/projects/4c06aa6e88634a97901bd41cf05c3bb2/users/2a1901d751fe459896a55219b1f2646f/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:39:01.979640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbaac99-fc00-4603-82ce-f741c573a89e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:01.979925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbaac99-fc00-4603-82ce-f741c573a89e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:01.980204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbaac99-fc00-4603-82ce-f741c573a89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:01.988220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1065/2132] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:01 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:01.991708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e55db-4a5e-4cf5-b7da-74996dda8802 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.011432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bfbaac99-fc00-4603-82ce-f741c573a89e None None] MFA Rules not processed for user `2a1901d751fe459896a55219b1f2646f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:02.012683 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bfbaac99-fc00-4603-82ce-f741c573a89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:02.031344 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:02.059130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e55db-4a5e-4cf5-b7da-74996dda8802 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.063903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038ec10-07ec-4e43-8cd4-fb83e837c94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.065064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038ec10-07ec-4e43-8cd4-fb83e837c94a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.065351 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038ec10-07ec-4e43-8cd4-fb83e837c94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.065631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038ec10-07ec-4e43-8cd4-fb83e837c94a admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.066297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038ec10-07ec-4e43-8cd4-fb83e837c94a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.068250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038ec10-07ec-4e43-8cd4-fb83e837c94a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.084493 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1068/2133] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:02.089138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbaac99-fc00-4603-82ce-f741c573a89e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.092452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1066/2134] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:01 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:02.097022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038ec10-07ec-4e43-8cd4-fb83e837c94a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.100423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbaac99-fc00-4603-82ce-f741c573a89e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.106981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830d9c2-a335-468a-bffe-a43db71c22bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.107659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830d9c2-a335-468a-bffe-a43db71c22bc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.107785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038ec10-07ec-4e43-8cd4-fb83e837c94a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.107946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830d9c2-a335-468a-bffe-a43db71c22bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.108224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830d9c2-a335-468a-bffe-a43db71c22bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.108904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830d9c2-a335-468a-bffe-a43db71c22bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.111561 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830d9c2-a335-468a-bffe-a43db71c22bc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.113248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc4eac-59b5-4cbf-9e36-5f4029cdac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.113859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4eac-59b5-4cbf-9e36-5f4029cdac46 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.114140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4eac-59b5-4cbf-9e36-5f4029cdac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.114413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4eac-59b5-4cbf-9e36-5f4029cdac46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.115154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc4eac-59b5-4cbf-9e36-5f4029cdac46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.116728 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc4eac-59b5-4cbf-9e36-5f4029cdac46 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.133024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1067/2135] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:02.137008 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc4eac-59b5-4cbf-9e36-5f4029cdac46 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.147766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc4eac-59b5-4cbf-9e36-5f4029cdac46 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.148682 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1069/2136] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:02 2022] POST /identity/v3/projects => generated 409 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:02.152622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830d9c2-a335-468a-bffe-a43db71c22bc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.208577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5475fe-73bb-492b-81e4-2979aa1b6573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.209506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5475fe-73bb-492b-81e4-2979aa1b6573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.209788 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5475fe-73bb-492b-81e4-2979aa1b6573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.210064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5475fe-73bb-492b-81e4-2979aa1b6573 admin admin] PATH_INFO: `/v3/projects/c443b57057f24fcf877008bc91224b58/users/ff5bc807376b4692b73e09df31c3ca26/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.216301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830d9c2-a335-468a-bffe-a43db71c22bc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.222003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194807ca-c619-4a5c-b9ff-0b419de4b1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.222640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194807ca-c619-4a5c-b9ff-0b419de4b1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.222931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194807ca-c619-4a5c-b9ff-0b419de4b1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.223258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194807ca-c619-4a5c-b9ff-0b419de4b1b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.223941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194807ca-c619-4a5c-b9ff-0b419de4b1b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.226023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194807ca-c619-4a5c-b9ff-0b419de4b1b4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.229731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5475fe-73bb-492b-81e4-2979aa1b6573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c443b57057f24fcf877008bc91224b58, user_id=ff5bc807376b4692b73e09df31c3ca26, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.231640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5475fe-73bb-492b-81e4-2979aa1b6573 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.247252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1068/2137] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:02 2022] PUT /identity/v3/projects/c443b57057f24fcf877008bc91224b58/users/ff5bc807376b4692b73e09df31c3ca26/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:02.253964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1070/2138] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:02 2022] POST /identity/v3/users => generated 311 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:02.254100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5475fe-73bb-492b-81e4-2979aa1b6573 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.255306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194807ca-c619-4a5c-b9ff-0b419de4b1b4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.265122 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194807ca-c619-4a5c-b9ff-0b419de4b1b4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.273385 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:02.320151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5475fe-73bb-492b-81e4-2979aa1b6573 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.326910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e66e5-9c84-426c-aba8-e881050764a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.327594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e66e5-9c84-426c-aba8-e881050764a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.327885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e66e5-9c84-426c-aba8-e881050764a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.328169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e66e5-9c84-426c-aba8-e881050764a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.328881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e66e5-9c84-426c-aba8-e881050764a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.330913 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e66e5-9c84-426c-aba8-e881050764a9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.337629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a0822-7e5e-4b26-864d-1ff025501489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.338269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a0822-7e5e-4b26-864d-1ff025501489 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.341358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a0822-7e5e-4b26-864d-1ff025501489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.341683 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a0822-7e5e-4b26-864d-1ff025501489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.342588 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a0822-7e5e-4b26-864d-1ff025501489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.344381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a0822-7e5e-4b26-864d-1ff025501489 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.347770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1069/2139] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:39:02.352164 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e66e5-9c84-426c-aba8-e881050764a9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.360032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1071/2140] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:02.362295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a0822-7e5e-4b26-864d-1ff025501489 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.364849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e66e5-9c84-426c-aba8-e881050764a9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.370331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eff6ae-cad1-45f6-b445-47bef39293f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.371009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eff6ae-cad1-45f6-b445-47bef39293f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.371339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eff6ae-cad1-45f6-b445-47bef39293f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.371626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eff6ae-cad1-45f6-b445-47bef39293f6 admin admin] PATH_INFO: `/v3/projects/c443b57057f24fcf877008bc91224b58/users/ff5bc807376b4692b73e09df31c3ca26/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.372704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a0822-7e5e-4b26-864d-1ff025501489 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.373625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eff6ae-cad1-45f6-b445-47bef39293f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c443b57057f24fcf877008bc91224b58, user_id=ff5bc807376b4692b73e09df31c3ca26, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.376332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eff6ae-cad1-45f6-b445-47bef39293f6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.378364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47412c64-1dbc-455b-b4ed-32c13d32d620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.379758 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47412c64-1dbc-455b-b4ed-32c13d32d620 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.380125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47412c64-1dbc-455b-b4ed-32c13d32d620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.380486 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47412c64-1dbc-455b-b4ed-32c13d32d620 admin admin] PATH_INFO: `/v3/projects/8a95fad44a4a4ed5b9a1b4c65243a810/users/422da118c03b4414bec93da2c20a486f/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.388032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1070/2141] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:02 2022] PUT /identity/v3/projects/c443b57057f24fcf877008bc91224b58/users/ff5bc807376b4692b73e09df31c3ca26/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:02.396898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47412c64-1dbc-455b-b4ed-32c13d32d620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a95fad44a4a4ed5b9a1b4c65243a810, user_id=422da118c03b4414bec93da2c20a486f, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.398490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c537886-5cbc-4fc1-a53e-1ed387b4e6b6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.398729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c537886-5cbc-4fc1-a53e-1ed387b4e6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.399009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c537886-5cbc-4fc1-a53e-1ed387b4e6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.399555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47412c64-1dbc-455b-b4ed-32c13d32d620 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.415563 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1072/2142] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:02 2022] PUT /identity/v3/projects/8a95fad44a4a4ed5b9a1b4c65243a810/users/422da118c03b4414bec93da2c20a486f/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:39:02.419342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47412c64-1dbc-455b-b4ed-32c13d32d620 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.432582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1c537886-5cbc-4fc1-a53e-1ed387b4e6b6 None None] MFA Rules not processed for user `ff5bc807376b4692b73e09df31c3ca26`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:02.433784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1c537886-5cbc-4fc1-a53e-1ed387b4e6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:02.442631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:02.489467 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47412c64-1dbc-455b-b4ed-32c13d32d620 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.494142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbab66d-83a0-4f88-9786-6c3687525317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.494754 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbab66d-83a0-4f88-9786-6c3687525317 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.495039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbab66d-83a0-4f88-9786-6c3687525317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.495382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbab66d-83a0-4f88-9786-6c3687525317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.496071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbab66d-83a0-4f88-9786-6c3687525317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.497698 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbab66d-83a0-4f88-9786-6c3687525317 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.506667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1071/2143] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:02.508935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1073/2144] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:39:02.512071 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c537886-5cbc-4fc1-a53e-1ed387b4e6b6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.518482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbab66d-83a0-4f88-9786-6c3687525317 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.520242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-abbab66d-83a0-4f88-9786-6c3687525317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:02.522459 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c537886-5cbc-4fc1-a53e-1ed387b4e6b6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.527148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634732d0-c18a-4e22-9040-fea0599bedbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.527800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634732d0-c18a-4e22-9040-fea0599bedbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.528096 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634732d0-c18a-4e22-9040-fea0599bedbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.528380 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634732d0-c18a-4e22-9040-fea0599bedbd admin admin] PATH_INFO: `/v3/projects/8a95fad44a4a4ed5b9a1b4c65243a810/users/422da118c03b4414bec93da2c20a486f/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.530321 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634732d0-c18a-4e22-9040-fea0599bedbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a95fad44a4a4ed5b9a1b4c65243a810, user_id=422da118c03b4414bec93da2c20a486f, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.533773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634732d0-c18a-4e22-9040-fea0599bedbd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.544208 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1072/2145] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:02 2022] PUT /identity/v3/projects/8a95fad44a4a4ed5b9a1b4c65243a810/users/422da118c03b4414bec93da2c20a486f/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:02.555420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c350b-6327-4ba4-8a1b-7510a51a0a18 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.555736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c350b-6327-4ba4-8a1b-7510a51a0a18 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.556017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c350b-6327-4ba4-8a1b-7510a51a0a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.582063 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbab66d-83a0-4f88-9786-6c3687525317 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.588283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1f3c350b-6327-4ba4-8a1b-7510a51a0a18 None None] MFA Rules not processed for user `422da118c03b4414bec93da2c20a486f`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:02.589492 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1f3c350b-6327-4ba4-8a1b-7510a51a0a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:02.617459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:02.640819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f950ef-50c6-4cc0-a55a-8d2665d4fb0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.641450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f950ef-50c6-4cc0-a55a-8d2665d4fb0c service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.641739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f950ef-50c6-4cc0-a55a-8d2665d4fb0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.642080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f950ef-50c6-4cc0-a55a-8d2665d4fb0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.669317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1073/2146] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:02 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:39:02.683164 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c350b-6327-4ba4-8a1b-7510a51a0a18 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.685837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1f3c350b-6327-4ba4-8a1b-7510a51a0a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:02.698745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c350b-6327-4ba4-8a1b-7510a51a0a18 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.702408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f4277-00e6-43b2-b86b-3ab4ce6bbc0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.703029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f4277-00e6-43b2-b86b-3ab4ce6bbc0b service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.703439 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f4277-00e6-43b2-b86b-3ab4ce6bbc0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.703816 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f4277-00e6-43b2-b86b-3ab4ce6bbc0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:02.712100 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f950ef-50c6-4cc0-a55a-8d2665d4fb0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.712971 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f4277-00e6-43b2-b86b-3ab4ce6bbc0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:02.713581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f950ef-50c6-4cc0-a55a-8d2665d4fb0c service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.714408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f4277-00e6-43b2-b86b-3ab4ce6bbc0b service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:02.724511 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1074/2147] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:02.724845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1074/2148] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:02 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:39:02.896427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f950ef-50c6-4cc0-a55a-8d2665d4fb0c service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:02.905292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f950ef-50c6-4cc0-a55a-8d2665d4fb0c service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:02.909953 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72ad6e-093b-4b82-8917-63d94fe2d2de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:02.910558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72ad6e-093b-4b82-8917-63d94fe2d2de service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:02.910840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72ad6e-093b-4b82-8917-63d94fe2d2de service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:02.911142 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72ad6e-093b-4b82-8917-63d94fe2d2de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:03.016717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72ad6e-093b-4b82-8917-63d94fe2d2de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:03.023465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72ad6e-093b-4b82-8917-63d94fe2d2de service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:03.036358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1075/2149] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:04.414743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f4277-00e6-43b2-b86b-3ab4ce6bbc0b service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:04.425881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f4277-00e6-43b2-b86b-3ab4ce6bbc0b service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:04.429929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca272cd-d736-485f-90f2-1428aea645a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:04.430715 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca272cd-d736-485f-90f2-1428aea645a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:04.431147 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca272cd-d736-485f-90f2-1428aea645a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:04.431470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca272cd-d736-485f-90f2-1428aea645a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:04.503830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca272cd-d736-485f-90f2-1428aea645a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:04.505584 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca272cd-d736-485f-90f2-1428aea645a3 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:04.515452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1075/2150] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:05.146431 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72ad6e-093b-4b82-8917-63d94fe2d2de service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:05.156794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72ad6e-093b-4b82-8917-63d94fe2d2de service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:05.161366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0d129-8ed7-41e0-9376-1167b29f7eb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:05.161972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0d129-8ed7-41e0-9376-1167b29f7eb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:05.162260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0d129-8ed7-41e0-9376-1167b29f7eb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:05.162537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0d129-8ed7-41e0-9376-1167b29f7eb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:05.223993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0d129-8ed7-41e0-9376-1167b29f7eb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:05.225438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0d129-8ed7-41e0-9376-1167b29f7eb0 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:05.234872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1076/2151] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:11.772742 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca272cd-d736-485f-90f2-1428aea645a3 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:11.796050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:11.885911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca272cd-d736-485f-90f2-1428aea645a3 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:11.890331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:11.890975 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:11.891283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:11.891562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] PATH_INFO: `/v3/users/2a1901d751fe459896a55219b1f2646f` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:11.893396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a1901d751fe459896a55219b1f2646f)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:11.895456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:11.975836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a1901d751fe459896a55219b1f2646f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:11.989716 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a1901d751fe459896a55219b1f2646f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:11.990466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a1901d751fe459896a55219b1f2646f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:11.999339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a1901d751fe459896a55219b1f2646f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.009457 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a1901d751fe459896a55219b1f2646f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.016452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1076/2152] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:11 2022] DELETE /identity/v3/users/2a1901d751fe459896a55219b1f2646f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:12.021262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aab44e-2a42-44fc-bce3-a09cbcfe4820 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:12.021808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aab44e-2a42-44fc-bce3-a09cbcfe4820 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:12.021808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aab44e-2a42-44fc-bce3-a09cbcfe4820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:12.044587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b5aab44e-2a42-44fc-bce3-a09cbcfe4820 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:12.045816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b5aab44e-2a42-44fc-bce3-a09cbcfe4820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:12.064172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:12.114772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1077/2153] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:12.124195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:12.125623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:12.198491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21194215-7a5b-4cc8-8640-c3caa79774cc admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:12.202410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed69f25-473f-41d5-bb55-626d92bfd7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:12.204561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed69f25-473f-41d5-bb55-626d92bfd7fe service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:12.204561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed69f25-473f-41d5-bb55-626d92bfd7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:12.204561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed69f25-473f-41d5-bb55-626d92bfd7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:12.216957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed69f25-473f-41d5-bb55-626d92bfd7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:12.218433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed69f25-473f-41d5-bb55-626d92bfd7fe service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:12.228847 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1077/2154] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:39:12.345570 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aab44e-2a42-44fc-bce3-a09cbcfe4820 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:12.351028 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b5aab44e-2a42-44fc-bce3-a09cbcfe4820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:12.364243 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aab44e-2a42-44fc-bce3-a09cbcfe4820 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:12.368995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:12.369662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:12.369946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:12.370226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] PATH_INFO: `/v3/users/e2202cff1cdb4005ab333557aa2d06bb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:12.371424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2202cff1cdb4005ab333557aa2d06bb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:12.373354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:12.423576 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2202cff1cdb4005ab333557aa2d06bb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.432063 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2202cff1cdb4005ab333557aa2d06bb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.446901 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2202cff1cdb4005ab333557aa2d06bb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.447667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2202cff1cdb4005ab333557aa2d06bb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.448318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2202cff1cdb4005ab333557aa2d06bb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.460691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1078/2155] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:12 2022] DELETE /identity/v3/users/e2202cff1cdb4005ab333557aa2d06bb => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:12.465075 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bb1e3-786d-44f8-b2c9-5f86a8ab8eb4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:12.465372 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bb1e3-786d-44f8-b2c9-5f86a8ab8eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:12.465659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bb1e3-786d-44f8-b2c9-5f86a8ab8eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:12.484719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b94bb1e3-786d-44f8-b2c9-5f86a8ab8eb4 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:12.485944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b94bb1e3-786d-44f8-b2c9-5f86a8ab8eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:12.506101 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:12.512093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:12.513886 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:12.564828 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1078/2156] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:12.574723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bb1e3-786d-44f8-b2c9-5f86a8ab8eb4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:12.576214 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b94bb1e3-786d-44f8-b2c9-5f86a8ab8eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:12.589239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebd0a2-222c-4da1-a638-21bbc8c1689d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:12.593933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:12.594579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:12.594861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:12.595166 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] PATH_INFO: `/v3/projects/4c06aa6e88634a97901bd41cf05c3bb2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:12.596934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c06aa6e88634a97901bd41cf05c3bb2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:12.599126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:12.637423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bb1e3-786d-44f8-b2c9-5f86a8ab8eb4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:12.637959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:12.648049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c06aa6e88634a97901bd41cf05c3bb2', 'request_id': 'req-6aa916af-eb0a-4b4a-888d-80f8649d198b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.655309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c06aa6e88634a97901bd41cf05c3bb2', 'request_id': 'req-6aa916af-eb0a-4b4a-888d-80f8649d198b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.661747 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c06aa6e88634a97901bd41cf05c3bb2', 'request_id': 'req-6aa916af-eb0a-4b4a-888d-80f8649d198b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.673186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] The token cache is being invalidate because project 4c06aa6e88634a97901bd41cf05c3bb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:12.673475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.674773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.676549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1079/2157] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:12 2022] DELETE /identity/v3/projects/4c06aa6e88634a97901bd41cf05c3bb2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:39:12.680161 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:12.681560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:12.687690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8b1a8-e1d0-4f21-b949-449086460f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:12.688320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8b1a8-e1d0-4f21-b949-449086460f7e service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:12.688607 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8b1a8-e1d0-4f21-b949-449086460f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:12.688890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8b1a8-e1d0-4f21-b949-449086460f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:12.691461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9ad8b1a8-e1d0-4f21-b949-449086460f7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:12.747805 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa916af-eb0a-4b4a-888d-80f8649d198b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:12.753114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:12.753807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:12.754088 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:12.754628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] PATH_INFO: `/v3/users/422da118c03b4414bec93da2c20a486f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:12.755577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8b1a8-e1d0-4f21-b949-449086460f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:12.755812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=422da118c03b4414bec93da2c20a486f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:12.757024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8b1a8-e1d0-4f21-b949-449086460f7e service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:12.757705 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:12.766537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1079/2158] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:12.809437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '422da118c03b4414bec93da2c20a486f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.817631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '422da118c03b4414bec93da2c20a486f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.834005 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '422da118c03b4414bec93da2c20a486f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.834850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '422da118c03b4414bec93da2c20a486f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.835744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '422da118c03b4414bec93da2c20a486f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:12.851690 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1080/2159] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:12 2022] DELETE /identity/v3/users/422da118c03b4414bec93da2c20a486f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:39:13.070478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8b1a8-e1d0-4f21-b949-449086460f7e service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:13.073739 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9ad8b1a8-e1d0-4f21-b949-449086460f7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:13.092764 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8b1a8-e1d0-4f21-b949-449086460f7e service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:13.131870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:13.163491 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:13.164274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:13.164564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:13.164846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] PATH_INFO: `/v3/projects/41f2241bc21f4a20a00a8a7e4b52e36c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:13.166153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f2241bc21f4a20a00a8a7e4b52e36c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:13.173479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:13.233048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f2241bc21f4a20a00a8a7e4b52e36c', 'request_id': 'req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.244482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f2241bc21f4a20a00a8a7e4b52e36c', 'request_id': 'req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.251677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f2241bc21f4a20a00a8a7e4b52e36c', 'request_id': 'req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.265930 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:13.267585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:13.269575 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] The token cache is being invalidate because project 41f2241bc21f4a20a00a8a7e4b52e36c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:13.269875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.270502 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.274809 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1080/2160] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:13 2022] DELETE /identity/v3/projects/41f2241bc21f4a20a00a8a7e4b52e36c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:13.275749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:13.277168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:13.282395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20553766-677e-4718-aa54-0631711a10a9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:13.355451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8004ffb-fedc-4e7c-a1f6-fd4189bb452f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:13.358863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:13.364722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384f0b3-5de3-4048-86ad-82b0785b31ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:13.365697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384f0b3-5de3-4048-86ad-82b0785b31ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:13.366114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384f0b3-5de3-4048-86ad-82b0785b31ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:13.366506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384f0b3-5de3-4048-86ad-82b0785b31ea admin admin] PATH_INFO: `/v3/users/3455fb5f83b541fab52dd8078684791f` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:13.369005 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384f0b3-5de3-4048-86ad-82b0785b31ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=3455fb5f83b541fab52dd8078684791f)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:13.371442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384f0b3-5de3-4048-86ad-82b0785b31ea admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:13.386293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47677d-588c-4708-aa2f-f62a3f95f7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:13.386961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47677d-588c-4708-aa2f-f62a3f95f7e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:13.387472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47677d-588c-4708-aa2f-f62a3f95f7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:13.387885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47677d-588c-4708-aa2f-f62a3f95f7e7 admin admin] PATH_INFO: `/v3/projects/8a95fad44a4a4ed5b9a1b4c65243a810` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:13.390215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47677d-588c-4708-aa2f-f62a3f95f7e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a95fad44a4a4ed5b9a1b4c65243a810)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:13.392138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47677d-588c-4708-aa2f-f62a3f95f7e7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:13.434585 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7384f0b3-5de3-4048-86ad-82b0785b31ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3455fb5f83b541fab52dd8078684791f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.441711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0d47677d-588c-4708-aa2f-f62a3f95f7e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a95fad44a4a4ed5b9a1b4c65243a810', 'request_id': 'req-0d47677d-588c-4708-aa2f-f62a3f95f7e7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.445997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:13.447289 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7384f0b3-5de3-4048-86ad-82b0785b31ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3455fb5f83b541fab52dd8078684791f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.448673 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7384f0b3-5de3-4048-86ad-82b0785b31ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3455fb5f83b541fab52dd8078684791f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.449443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0d47677d-588c-4708-aa2f-f62a3f95f7e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a95fad44a4a4ed5b9a1b4c65243a810', 'request_id': 'req-0d47677d-588c-4708-aa2f-f62a3f95f7e7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.456581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0d47677d-588c-4708-aa2f-f62a3f95f7e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a95fad44a4a4ed5b9a1b4c65243a810', 'request_id': 'req-0d47677d-588c-4708-aa2f-f62a3f95f7e7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.458497 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7384f0b3-5de3-4048-86ad-82b0785b31ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3455fb5f83b541fab52dd8078684791f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.465831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-7384f0b3-5de3-4048-86ad-82b0785b31ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3455fb5f83b541fab52dd8078684791f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.467761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0d47677d-588c-4708-aa2f-f62a3f95f7e7 admin admin] The token cache is being invalidate because project 8a95fad44a4a4ed5b9a1b4c65243a810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:13.468057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0d47677d-588c-4708-aa2f-f62a3f95f7e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.468699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0d47677d-588c-4708-aa2f-f62a3f95f7e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.470288 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1081/2161] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:13 2022] DELETE /identity/v3/projects/8a95fad44a4a4ed5b9a1b4c65243a810 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:13.472614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1081/2162] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:13 2022] DELETE /identity/v3/users/3455fb5f83b541fab52dd8078684791f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:13.475630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ac801-7378-4b0c-aa20-ba461c7a38e5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:13.476144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ac801-7378-4b0c-aa20-ba461c7a38e5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:13.476533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ac801-7378-4b0c-aa20-ba461c7a38e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:13.497607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d10ac801-7378-4b0c-aa20-ba461c7a38e5 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:13.498820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d10ac801-7378-4b0c-aa20-ba461c7a38e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:13.563342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1082/2163] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:13.573656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5d21f-45e1-43fd-90b3-ac2406035c50 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:13.573971 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5d21f-45e1-43fd-90b3-ac2406035c50 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:13.574253 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5d21f-45e1-43fd-90b3-ac2406035c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:13.593953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f9d5d21f-45e1-43fd-90b3-ac2406035c50 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:13.595299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f9d5d21f-45e1-43fd-90b3-ac2406035c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:13.604477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1082/2164] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:13.608986 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ac801-7378-4b0c-aa20-ba461c7a38e5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:13.620279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ac801-7378-4b0c-aa20-ba461c7a38e5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:13.625120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454330a6-cfad-46b9-b90e-2f7f2e2fd4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:13.625818 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454330a6-cfad-46b9-b90e-2f7f2e2fd4dc admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:13.626173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454330a6-cfad-46b9-b90e-2f7f2e2fd4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:13.626522 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454330a6-cfad-46b9-b90e-2f7f2e2fd4dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:13.627348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454330a6-cfad-46b9-b90e-2f7f2e2fd4dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:39:13.627802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454330a6-cfad-46b9-b90e-2f7f2e2fd4dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:13.630452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454330a6-cfad-46b9-b90e-2f7f2e2fd4dc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:13.642058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1083/2165] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:39:13.646616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5d21f-45e1-43fd-90b3-ac2406035c50 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:13.655956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5d21f-45e1-43fd-90b3-ac2406035c50 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:13.661210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edc7ce-7d0f-4257-ac0c-b480bf29fdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:13.661827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edc7ce-7d0f-4257-ac0c-b480bf29fdf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:13.662113 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edc7ce-7d0f-4257-ac0c-b480bf29fdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:13.662391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edc7ce-7d0f-4257-ac0c-b480bf29fdf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:13.663090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edc7ce-7d0f-4257-ac0c-b480bf29fdf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:13.665027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edc7ce-7d0f-4257-ac0c-b480bf29fdf8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:13.695272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1083/2166] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:13 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:13.700578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454330a6-cfad-46b9-b90e-2f7f2e2fd4dc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:13.711453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454330a6-cfad-46b9-b90e-2f7f2e2fd4dc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:13.766451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edc7ce-7d0f-4257-ac0c-b480bf29fdf8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:13.768754 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d7edc7ce-7d0f-4257-ac0c-b480bf29fdf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:13.774907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816f9e7-9890-425b-ae96-ab07dda153b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:13.775590 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816f9e7-9890-425b-ae96-ab07dda153b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:13.775891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816f9e7-9890-425b-ae96-ab07dda153b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:13.776277 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816f9e7-9890-425b-ae96-ab07dda153b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:13.776814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816f9e7-9890-425b-ae96-ab07dda153b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:13.779275 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816f9e7-9890-425b-ae96-ab07dda153b0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:13.788949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:13.808975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1084/2167] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:13 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:13.813185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816f9e7-9890-425b-ae96-ab07dda153b0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:13.823461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816f9e7-9890-425b-ae96-ab07dda153b0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:13.828095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe54897-8b40-49b7-bdbf-d8b0f51404ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:13.828696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe54897-8b40-49b7-bdbf-d8b0f51404ed admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:13.828975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe54897-8b40-49b7-bdbf-d8b0f51404ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:13.829249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe54897-8b40-49b7-bdbf-d8b0f51404ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:13.829962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe54897-8b40-49b7-bdbf-d8b0f51404ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:13.831681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe54897-8b40-49b7-bdbf-d8b0f51404ed admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:13.842161 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1085/2168] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:39:13.844609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edc7ce-7d0f-4257-ac0c-b480bf29fdf8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:13.846813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe54897-8b40-49b7-bdbf-d8b0f51404ed admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:13.850430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:13.851128 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:13.851418 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:13.851908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] PATH_INFO: `/v3/projects/4202efd1004b421cb58de3f8a9093425` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:13.853715 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4202efd1004b421cb58de3f8a9093425)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:13.855701 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:13.856619 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe54897-8b40-49b7-bdbf-d8b0f51404ed admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:13.862210 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc009f48-3f7b-4d8f-8430-ab7ade85d973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:13.862920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc009f48-3f7b-4d8f-8430-ab7ade85d973 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:13.863238 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc009f48-3f7b-4d8f-8430-ab7ade85d973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:13.863514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc009f48-3f7b-4d8f-8430-ab7ade85d973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:13.864192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc009f48-3f7b-4d8f-8430-ab7ade85d973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:13.865781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc009f48-3f7b-4d8f-8430-ab7ade85d973 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:13.875805 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1086/2169] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:39:13.881060 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc009f48-3f7b-4d8f-8430-ab7ade85d973 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:13.891019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc009f48-3f7b-4d8f-8430-ab7ade85d973 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:13.904617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4202efd1004b421cb58de3f8a9093425', 'request_id': 'req-a0a71780-a023-494f-85fc-8b444ce83014'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.910549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4202efd1004b421cb58de3f8a9093425', 'request_id': 'req-a0a71780-a023-494f-85fc-8b444ce83014'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.916738 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4202efd1004b421cb58de3f8a9093425', 'request_id': 'req-a0a71780-a023-494f-85fc-8b444ce83014'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.933918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] The token cache is being invalidate because project 4202efd1004b421cb58de3f8a9093425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:13.933918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.934932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:13.936808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1084/2170] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:13 2022] DELETE /identity/v3/projects/4202efd1004b421cb58de3f8a9093425 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:39:13.940879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:13.942347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:13.958565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04687f62-5f17-488d-896b-1c8b4189c802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:13.959473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04687f62-5f17-488d-896b-1c8b4189c802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:13.959928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04687f62-5f17-488d-896b-1c8b4189c802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:13.960340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04687f62-5f17-488d-896b-1c8b4189c802 admin admin] PATH_INFO: `/v3/projects/0c38c706aa4d4edd99d16bb84302512c/users/96e32ebbde6c43858e747b2e9d26e290/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:13.981208 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04687f62-5f17-488d-896b-1c8b4189c802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c38c706aa4d4edd99d16bb84302512c, user_id=96e32ebbde6c43858e747b2e9d26e290, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:13.983926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04687f62-5f17-488d-896b-1c8b4189c802 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:13.995538 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:14.001276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1087/2171] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:13 2022] PUT /identity/v3/projects/0c38c706aa4d4edd99d16bb84302512c/users/96e32ebbde6c43858e747b2e9d26e290/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:14.012017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542493d1-32c6-4b8e-aab1-9c8d7076a13e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.012307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542493d1-32c6-4b8e-aab1-9c8d7076a13e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.012607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542493d1-32c6-4b8e-aab1-9c8d7076a13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.020802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a71780-a023-494f-85fc-8b444ce83014 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.045117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-542493d1-32c6-4b8e-aab1-9c8d7076a13e None None] MFA Rules not processed for user `96e32ebbde6c43858e747b2e9d26e290`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:14.046371 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-542493d1-32c6-4b8e-aab1-9c8d7076a13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:14.088532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.089215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.089485 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.089756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] PATH_INFO: `/v3/users/ff5bc807376b4692b73e09df31c3ca26` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.090956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff5bc807376b4692b73e09df31c3ca26)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.093078 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.135224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1088/2172] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:14.136190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542493d1-32c6-4b8e-aab1-9c8d7076a13e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.138014 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-542493d1-32c6-4b8e-aab1-9c8d7076a13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:14.155178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542493d1-32c6-4b8e-aab1-9c8d7076a13e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.157154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff5bc807376b4692b73e09df31c3ca26'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:14.161759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829875b1-4ef8-4e2a-a564-d2a53cfd787e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.162700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829875b1-4ef8-4e2a-a564-d2a53cfd787e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.163254 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829875b1-4ef8-4e2a-a564-d2a53cfd787e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.163670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829875b1-4ef8-4e2a-a564-d2a53cfd787e admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.164376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829875b1-4ef8-4e2a-a564-d2a53cfd787e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.166430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829875b1-4ef8-4e2a-a564-d2a53cfd787e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.170955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff5bc807376b4692b73e09df31c3ca26'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:14.172463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff5bc807376b4692b73e09df31c3ca26'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:14.181638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff5bc807376b4692b73e09df31c3ca26'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:14.191135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff5bc807376b4692b73e09df31c3ca26'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:14.197404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1085/2173] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:13 2022] DELETE /identity/v3/users/ff5bc807376b4692b73e09df31c3ca26 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:39:14.200556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1089/2174] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:14 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:14.202940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.205374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:14.280171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb3e94-1440-484b-9aaf-1deb7a281b50 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.291853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:14.341217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8764f-fbdb-4d41-95cc-e27f944e2c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.341847 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8764f-fbdb-4d41-95cc-e27f944e2c80 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.342136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8764f-fbdb-4d41-95cc-e27f944e2c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.342428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8764f-fbdb-4d41-95cc-e27f944e2c80 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.343137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8764f-fbdb-4d41-95cc-e27f944e2c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.345088 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8764f-fbdb-4d41-95cc-e27f944e2c80 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.371394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1086/2175] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:14 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:14.373796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829875b1-4ef8-4e2a-a564-d2a53cfd787e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.384672 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829875b1-4ef8-4e2a-a564-d2a53cfd787e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.389326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2b4fc-1cfb-4b8d-84a2-72b3b3be8952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.389946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2b4fc-1cfb-4b8d-84a2-72b3b3be8952 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.390234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2b4fc-1cfb-4b8d-84a2-72b3b3be8952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.390512 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2b4fc-1cfb-4b8d-84a2-72b3b3be8952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.391245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2b4fc-1cfb-4b8d-84a2-72b3b3be8952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.392897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2b4fc-1cfb-4b8d-84a2-72b3b3be8952 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.404514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1090/2176] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:14.409549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8764f-fbdb-4d41-95cc-e27f944e2c80 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.419901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8764f-fbdb-4d41-95cc-e27f944e2c80 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.425463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b9b37-1618-4201-bc50-93e110164ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.425991 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b9b37-1618-4201-bc50-93e110164ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.426272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b9b37-1618-4201-bc50-93e110164ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.426545 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b9b37-1618-4201-bc50-93e110164ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.427766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b9b37-1618-4201-bc50-93e110164ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.429011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b9b37-1618-4201-bc50-93e110164ef0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.443739 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1087/2177] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:14.448199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2b4fc-1cfb-4b8d-84a2-72b3b3be8952 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.460761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2b4fc-1cfb-4b8d-84a2-72b3b3be8952 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.466079 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc3f09-5f1e-40bc-acb2-5f6f2e4f15c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.466821 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3f09-5f1e-40bc-acb2-5f6f2e4f15c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.467260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3f09-5f1e-40bc-acb2-5f6f2e4f15c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.467613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3f09-5f1e-40bc-acb2-5f6f2e4f15c5 admin admin] PATH_INFO: `/v3/projects/ffc815b360e14d08bc0a00053fb2f518/users/41468b9023024d0391dd9d8fefbbebb6/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.476729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:14.485022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc3f09-5f1e-40bc-acb2-5f6f2e4f15c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc815b360e14d08bc0a00053fb2f518, user_id=41468b9023024d0391dd9d8fefbbebb6, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.488395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc3f09-5f1e-40bc-acb2-5f6f2e4f15c5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.505701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1091/2178] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:14 2022] PUT /identity/v3/projects/ffc815b360e14d08bc0a00053fb2f518/users/41468b9023024d0391dd9d8fefbbebb6/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:14.518681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc3f09-5f1e-40bc-acb2-5f6f2e4f15c5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.518998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2a0f-09ba-4810-87be-7c1c665eab82 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.519727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2a0f-09ba-4810-87be-7c1c665eab82 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.520021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2a0f-09ba-4810-87be-7c1c665eab82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.521418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-02bc3f09-5f1e-40bc-acb2-5f6f2e4f15c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:14.553443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-69bc2a0f-09ba-4810-87be-7c1c665eab82 None None] MFA Rules not processed for user `41468b9023024d0391dd9d8fefbbebb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:14.554822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-69bc2a0f-09ba-4810-87be-7c1c665eab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:14.607907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc3f09-5f1e-40bc-acb2-5f6f2e4f15c5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.613184 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbd32c-5dcb-4642-b201-ce4abf18de2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.613824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbd32c-5dcb-4642-b201-ce4abf18de2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.614111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbd32c-5dcb-4642-b201-ce4abf18de2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.614391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbd32c-5dcb-4642-b201-ce4abf18de2e admin admin] PATH_INFO: `/v3/projects/c443b57057f24fcf877008bc91224b58` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.615698 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbd32c-5dcb-4642-b201-ce4abf18de2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c443b57057f24fcf877008bc91224b58)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.618334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbd32c-5dcb-4642-b201-ce4abf18de2e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.641599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1088/2179] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:14.647056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2a0f-09ba-4810-87be-7c1c665eab82 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.659226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2a0f-09ba-4810-87be-7c1c665eab82 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.667749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-50bbd32c-5dcb-4642-b201-ce4abf18de2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c443b57057f24fcf877008bc91224b58', 'request_id': 'req-50bbd32c-5dcb-4642-b201-ce4abf18de2e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:14.676384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-50bbd32c-5dcb-4642-b201-ce4abf18de2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c443b57057f24fcf877008bc91224b58', 'request_id': 'req-50bbd32c-5dcb-4642-b201-ce4abf18de2e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:14.682092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-50bbd32c-5dcb-4642-b201-ce4abf18de2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c443b57057f24fcf877008bc91224b58', 'request_id': 'req-50bbd32c-5dcb-4642-b201-ce4abf18de2e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:14.694272 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-50bbd32c-5dcb-4642-b201-ce4abf18de2e admin admin] The token cache is being invalidate because project c443b57057f24fcf877008bc91224b58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:14.694559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-50bbd32c-5dcb-4642-b201-ce4abf18de2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:14.695260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-50bbd32c-5dcb-4642-b201-ce4abf18de2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:14.696821 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1092/2180] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:14 2022] DELETE /identity/v3/projects/c443b57057f24fcf877008bc91224b58 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:14.701577 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773ea9b-5476-4f7c-8558-7806af54e20b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.701897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773ea9b-5476-4f7c-8558-7806af54e20b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.702181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773ea9b-5476-4f7c-8558-7806af54e20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.719324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b6760-1e8c-4f63-b9d1-e273c76e1672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.719947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b6760-1e8c-4f63-b9d1-e273c76e1672 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.720227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b6760-1e8c-4f63-b9d1-e273c76e1672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.720503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b6760-1e8c-4f63-b9d1-e273c76e1672 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.721852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b6760-1e8c-4f63-b9d1-e273c76e1672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.722884 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d773ea9b-5476-4f7c-8558-7806af54e20b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:14.724054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b6760-1e8c-4f63-b9d1-e273c76e1672 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.724170 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d773ea9b-5476-4f7c-8558-7806af54e20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:14.733016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1093/2181] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:14.740964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d719ca-c06b-4151-8f11-8a5550c3394d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.741251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d719ca-c06b-4151-8f11-8a5550c3394d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.741527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d719ca-c06b-4151-8f11-8a5550c3394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.752080 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1089/2182] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:14 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:14.756239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b6760-1e8c-4f63-b9d1-e273c76e1672 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.757827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-be7b6760-1e8c-4f63-b9d1-e273c76e1672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:14.765538 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a2d719ca-c06b-4151-8f11-8a5550c3394d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:14.766711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a2d719ca-c06b-4151-8f11-8a5550c3394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:14.768328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:14.825794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1094/2183] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:39:14.826591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b6760-1e8c-4f63-b9d1-e273c76e1672 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.829791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d719ca-c06b-4151-8f11-8a5550c3394d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.831823 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b351b9-6ead-4b13-b159-20809441d27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.832431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b351b9-6ead-4b13-b159-20809441d27a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.832714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b351b9-6ead-4b13-b159-20809441d27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.832991 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b351b9-6ead-4b13-b159-20809441d27a admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.833653 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b351b9-6ead-4b13-b159-20809441d27a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.835614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b351b9-6ead-4b13-b159-20809441d27a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.839105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d719ca-c06b-4151-8f11-8a5550c3394d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.845110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6907c47-5b69-4308-a5fe-90202e9d373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.846363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6907c47-5b69-4308-a5fe-90202e9d373c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.846648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6907c47-5b69-4308-a5fe-90202e9d373c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.846928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6907c47-5b69-4308-a5fe-90202e9d373c admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.847647 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6907c47-5b69-4308-a5fe-90202e9d373c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:39:14.848039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6907c47-5b69-4308-a5fe-90202e9d373c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.849701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6907c47-5b69-4308-a5fe-90202e9d373c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.852388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:14.861089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1090/2184] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:14 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:14.864018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1095/2185] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:39:14.865053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b351b9-6ead-4b13-b159-20809441d27a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.867424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6907c47-5b69-4308-a5fe-90202e9d373c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.874831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b351b9-6ead-4b13-b159-20809441d27a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.877299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6907c47-5b69-4308-a5fe-90202e9d373c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.880003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13774e5f-685d-4a52-a466-359763c5b6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.880593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13774e5f-685d-4a52-a466-359763c5b6fd admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.880873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13774e5f-685d-4a52-a466-359763c5b6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.881146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13774e5f-685d-4a52-a466-359763c5b6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.881836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13774e5f-685d-4a52-a466-359763c5b6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.882271 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f6241-f74f-4fe7-b994-8a86f834d05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.882866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f6241-f74f-4fe7-b994-8a86f834d05e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.883198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f6241-f74f-4fe7-b994-8a86f834d05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.883458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13774e5f-685d-4a52-a466-359763c5b6fd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.883559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f6241-f74f-4fe7-b994-8a86f834d05e admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.884132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f6241-f74f-4fe7-b994-8a86f834d05e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.886222 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f6241-f74f-4fe7-b994-8a86f834d05e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.894632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1091/2186] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:39:14.899039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13774e5f-685d-4a52-a466-359763c5b6fd admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.909725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13774e5f-685d-4a52-a466-359763c5b6fd admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:14.914232 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6ed9d-f463-46c0-971b-eeceb3a95725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:14.914363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1096/2187] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:14 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:14.914969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6ed9d-f463-46c0-971b-eeceb3a95725 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:14.915321 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6ed9d-f463-46c0-971b-eeceb3a95725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:14.915603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6ed9d-f463-46c0-971b-eeceb3a95725 admin admin] PATH_INFO: `/v3/projects/8593f3f2b3364ec68b80be5941f755ba/users/c2db3de1b0aa425c8d70254a8ca44983/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:14.918571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f6241-f74f-4fe7-b994-8a86f834d05e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.930790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6ed9d-f463-46c0-971b-eeceb3a95725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8593f3f2b3364ec68b80be5941f755ba, user_id=c2db3de1b0aa425c8d70254a8ca44983, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:14.933475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6ed9d-f463-46c0-971b-eeceb3a95725 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:14.948702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1092/2188] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:14 2022] PUT /identity/v3/projects/8593f3f2b3364ec68b80be5941f755ba/users/c2db3de1b0aa425c8d70254a8ca44983/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:39:14.953561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6ed9d-f463-46c0-971b-eeceb3a95725 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:14.989197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f6241-f74f-4fe7-b994-8a86f834d05e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:15.056967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6ed9d-f463-46c0-971b-eeceb3a95725 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:15.060624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf01c0-62c8-47dc-9e63-1cb9dbaf7767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:15.061684 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf01c0-62c8-47dc-9e63-1cb9dbaf7767 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:15.062247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf01c0-62c8-47dc-9e63-1cb9dbaf7767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:15.062581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54408bae-a868-45a4-9df3-a5e445b2397b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:15.062712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf01c0-62c8-47dc-9e63-1cb9dbaf7767 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:15.063371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54408bae-a868-45a4-9df3-a5e445b2397b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:15.063508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf01c0-62c8-47dc-9e63-1cb9dbaf7767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:15.063785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54408bae-a868-45a4-9df3-a5e445b2397b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:15.064138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54408bae-a868-45a4-9df3-a5e445b2397b admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:15.065583 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54408bae-a868-45a4-9df3-a5e445b2397b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:15.066970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf01c0-62c8-47dc-9e63-1cb9dbaf7767 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:15.067348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54408bae-a868-45a4-9df3-a5e445b2397b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:15.080629 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1093/2189] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:15.085896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54408bae-a868-45a4-9df3-a5e445b2397b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:15.098839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54408bae-a868-45a4-9df3-a5e445b2397b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:15.101514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1097/2190] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:14 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:15.104835 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def22728-b6c1-40c0-82e0-faa52df96f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:15.105446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf01c0-62c8-47dc-9e63-1cb9dbaf7767 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:15.105574 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def22728-b6c1-40c0-82e0-faa52df96f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:15.105774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def22728-b6c1-40c0-82e0-faa52df96f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:15.106054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def22728-b6c1-40c0-82e0-faa52df96f70 admin admin] PATH_INFO: `/v3/projects/8593f3f2b3364ec68b80be5941f755ba/users/c2db3de1b0aa425c8d70254a8ca44983/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:15.108061 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def22728-b6c1-40c0-82e0-faa52df96f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8593f3f2b3364ec68b80be5941f755ba, user_id=c2db3de1b0aa425c8d70254a8ca44983, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:15.110848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def22728-b6c1-40c0-82e0-faa52df96f70 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:15.118787 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf01c0-62c8-47dc-9e63-1cb9dbaf7767 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:15.122866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1094/2191] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:15 2022] PUT /identity/v3/projects/8593f3f2b3364ec68b80be5941f755ba/users/c2db3de1b0aa425c8d70254a8ca44983/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:15.134521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad600fd-0cac-4c35-8eca-91146e6c84b6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:15.134837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad600fd-0cac-4c35-8eca-91146e6c84b6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:15.135164 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad600fd-0cac-4c35-8eca-91146e6c84b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:15.182566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-cad600fd-0cac-4c35-8eca-91146e6c84b6 None None] MFA Rules not processed for user `c2db3de1b0aa425c8d70254a8ca44983`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:15.184085 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cad600fd-0cac-4c35-8eca-91146e6c84b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:15.216633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfef98-9042-402b-8e98-d15e4596aacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:15.217260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfef98-9042-402b-8e98-d15e4596aacb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:15.217551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfef98-9042-402b-8e98-d15e4596aacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:15.217835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfef98-9042-402b-8e98-d15e4596aacb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:15.218595 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfef98-9042-402b-8e98-d15e4596aacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:15.220229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfef98-9042-402b-8e98-d15e4596aacb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:15.232216 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1098/2192] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:39:15.234054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:15.236006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfef98-9042-402b-8e98-d15e4596aacb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:15.248459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfef98-9042-402b-8e98-d15e4596aacb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:15.253324 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e7804-7cc3-43e3-9900-be1bfead9da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:15.254099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e7804-7cc3-43e3-9900-be1bfead9da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:15.254651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e7804-7cc3-43e3-9900-be1bfead9da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:15.255118 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e7804-7cc3-43e3-9900-be1bfead9da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:15.255693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e7804-7cc3-43e3-9900-be1bfead9da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:15.257279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e7804-7cc3-43e3-9900-be1bfead9da5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:15.258723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:15.270633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1099/2193] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:39:15.275323 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e7804-7cc3-43e3-9900-be1bfead9da5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:15.281180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1095/2194] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:15.290605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e7804-7cc3-43e3-9900-be1bfead9da5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:15.295780 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad600fd-0cac-4c35-8eca-91146e6c84b6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:15.296834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e0453-3e9c-4e0e-9568-7d1105cbe291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:15.297472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cad600fd-0cac-4c35-8eca-91146e6c84b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:15.297999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e0453-3e9c-4e0e-9568-7d1105cbe291 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:15.298456 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e0453-3e9c-4e0e-9568-7d1105cbe291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:15.299586 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e0453-3e9c-4e0e-9568-7d1105cbe291 admin admin] PATH_INFO: `/v3/projects/0b13f6a12091447da7be7bde4169a139/users/7d52bd0da53c4423be51f3b179ac20f9/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:15.317990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e0453-3e9c-4e0e-9568-7d1105cbe291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b13f6a12091447da7be7bde4169a139, user_id=7d52bd0da53c4423be51f3b179ac20f9, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:15.321749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e0453-3e9c-4e0e-9568-7d1105cbe291 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:15.340379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1100/2195] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:15 2022] PUT /identity/v3/projects/0b13f6a12091447da7be7bde4169a139/users/7d52bd0da53c4423be51f3b179ac20f9/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:39:15.353017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9c247-0b83-49ce-a433-fd669708c999 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:15.353397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9c247-0b83-49ce-a433-fd669708c999 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:15.353823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9c247-0b83-49ce-a433-fd669708c999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:15.368903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad600fd-0cac-4c35-8eca-91146e6c84b6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:15.386769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-96f9c247-0b83-49ce-a433-fd669708c999 None None] MFA Rules not processed for user `7d52bd0da53c4423be51f3b179ac20f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:15.388052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-96f9c247-0b83-49ce-a433-fd669708c999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:15.433482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6f0b4-26f5-4d4f-966a-6ce2b8a887af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:15.434400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6f0b4-26f5-4d4f-966a-6ce2b8a887af service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:15.434677 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6f0b4-26f5-4d4f-966a-6ce2b8a887af service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:15.434951 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6f0b4-26f5-4d4f-966a-6ce2b8a887af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:15.460297 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1101/2196] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:39:15.507689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6f0b4-26f5-4d4f-966a-6ce2b8a887af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:15.509964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6f0b4-26f5-4d4f-966a-6ce2b8a887af service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:15.519184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1096/2197] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:39:15.563524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9c247-0b83-49ce-a433-fd669708c999 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:15.574380 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9c247-0b83-49ce-a433-fd669708c999 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:15.578236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29988eae-6870-4178-8e24-12e53015fafa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:15.579013 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29988eae-6870-4178-8e24-12e53015fafa service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:15.579407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29988eae-6870-4178-8e24-12e53015fafa service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:15.580353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29988eae-6870-4178-8e24-12e53015fafa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:15.589151 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29988eae-6870-4178-8e24-12e53015fafa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:15.590578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29988eae-6870-4178-8e24-12e53015fafa service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:15.602352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1102/2198] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:15.712370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6f0b4-26f5-4d4f-966a-6ce2b8a887af service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:15.722813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6f0b4-26f5-4d4f-966a-6ce2b8a887af service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:15.731543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7abbe-d0de-407f-8844-72cec8904e65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:15.732228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7abbe-d0de-407f-8844-72cec8904e65 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:15.732541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7abbe-d0de-407f-8844-72cec8904e65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:15.732772 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7abbe-d0de-407f-8844-72cec8904e65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:15.735470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dfb7abbe-d0de-407f-8844-72cec8904e65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:15.774587 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:15.804259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7abbe-d0de-407f-8844-72cec8904e65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:15.806079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7abbe-d0de-407f-8844-72cec8904e65 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:15.817156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1097/2199] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:16.395125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e232c-f369-4b65-ac70-8f97cd21d52e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:16.395660 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e232c-f369-4b65-ac70-8f97cd21d52e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:16.395827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e232c-f369-4b65-ac70-8f97cd21d52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:16.418131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-243e232c-f369-4b65-ac70-8f97cd21d52e None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:16.419521 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-243e232c-f369-4b65-ac70-8f97cd21d52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:16.519993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1103/2200] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:39:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:16.567013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7abbe-d0de-407f-8844-72cec8904e65 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:16.568582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dfb7abbe-d0de-407f-8844-72cec8904e65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:16.651099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7abbe-d0de-407f-8844-72cec8904e65 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:16.656466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537bd32-798a-48dd-871b-71a8a0d959a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:16.657120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537bd32-798a-48dd-871b-71a8a0d959a7 service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:16.657407 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537bd32-798a-48dd-871b-71a8a0d959a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:16.657687 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537bd32-798a-48dd-871b-71a8a0d959a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:16.668038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537bd32-798a-48dd-871b-71a8a0d959a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:16.669662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537bd32-798a-48dd-871b-71a8a0d959a7 service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:16.678562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1098/2201] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:39:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:39:19.410400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e232c-f369-4b65-ac70-8f97cd21d52e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:19.480802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:19.503736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e232c-f369-4b65-ac70-8f97cd21d52e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:19.515278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:19.516293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:19.517078 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:19.518382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] PATH_INFO: `/v3/users/96e32ebbde6c43858e747b2e9d26e290` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:19.523116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e32ebbde6c43858e747b2e9d26e290)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:19.525955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:19.589433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e32ebbde6c43858e747b2e9d26e290'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:19.601735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e32ebbde6c43858e747b2e9d26e290'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:19.617549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e32ebbde6c43858e747b2e9d26e290'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:19.618586 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e32ebbde6c43858e747b2e9d26e290'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:19.619228 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e32ebbde6c43858e747b2e9d26e290'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:19.636004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1104/2202] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:19 2022] DELETE /identity/v3/users/96e32ebbde6c43858e747b2e9d26e290 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:19.641682 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f01e66-6152-4ae1-b538-ff139033de2c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:19.642286 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f01e66-6152-4ae1-b538-ff139033de2c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:19.642286 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f01e66-6152-4ae1-b538-ff139033de2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:19.673292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-73f01e66-6152-4ae1-b538-ff139033de2c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:19.675705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-73f01e66-6152-4ae1-b538-ff139033de2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:19.762592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1099/2203] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:39:19.772761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:19.774164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:19.841964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a50ba9-bde0-4b30-9b65-0aa812ee7307 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:19.845797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69dbb8-158f-44c9-b773-eaf8c4f90c5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:19.846425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69dbb8-158f-44c9-b773-eaf8c4f90c5d service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:19.846762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69dbb8-158f-44c9-b773-eaf8c4f90c5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:19.847044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69dbb8-158f-44c9-b773-eaf8c4f90c5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:19.857408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69dbb8-158f-44c9-b773-eaf8c4f90c5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:19.859252 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69dbb8-158f-44c9-b773-eaf8c4f90c5d service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:19.872541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1105/2204] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:20.196580 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f01e66-6152-4ae1-b538-ff139033de2c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:20.198338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-73f01e66-6152-4ae1-b538-ff139033de2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:20.218086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f01e66-6152-4ae1-b538-ff139033de2c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:20.223487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:20.224267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:20.224705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:20.225019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] PATH_INFO: `/v3/projects/0c38c706aa4d4edd99d16bb84302512c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:20.226380 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c38c706aa4d4edd99d16bb84302512c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:20.228726 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:20.284989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c38c706aa4d4edd99d16bb84302512c', 'request_id': 'req-4dcae8df-a5df-4be7-b159-d0477405e464'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:20.293885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c38c706aa4d4edd99d16bb84302512c', 'request_id': 'req-4dcae8df-a5df-4be7-b159-d0477405e464'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:20.296291 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:20.309320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c38c706aa4d4edd99d16bb84302512c', 'request_id': 'req-4dcae8df-a5df-4be7-b159-d0477405e464'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:20.342546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] The token cache is being invalidate because project 0c38c706aa4d4edd99d16bb84302512c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:20.343401 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:20.344555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:20.347175 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1100/2205] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:20 2022] DELETE /identity/v3/projects/0c38c706aa4d4edd99d16bb84302512c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:20.352629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69dbb8-158f-44c9-b773-eaf8c4f90c5d service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:20.374796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69dbb8-158f-44c9-b773-eaf8c4f90c5d service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:20.499263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:20.499263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:20.499263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:20.499263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] PATH_INFO: `/v3/users/41468b9023024d0391dd9d8fefbbebb6` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:20.501036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41468b9023024d0391dd9d8fefbbebb6)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:20.505087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:20.527761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:20.584207 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41468b9023024d0391dd9d8fefbbebb6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:20.594144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41468b9023024d0391dd9d8fefbbebb6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:20.617656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41468b9023024d0391dd9d8fefbbebb6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:20.619513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41468b9023024d0391dd9d8fefbbebb6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:20.620750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41468b9023024d0391dd9d8fefbbebb6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:20.656128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1106/2206] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:20 2022] DELETE /identity/v3/users/41468b9023024d0391dd9d8fefbbebb6 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:21.237371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:21.238937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:21.258419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcae8df-a5df-4be7-b159-d0477405e464 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:21.351823 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:21.353042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:21.353324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:21.353697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] PATH_INFO: `/v3/projects/ffc815b360e14d08bc0a00053fb2f518` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:21.356461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc815b360e14d08bc0a00053fb2f518)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:21.359936 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:21.507294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc815b360e14d08bc0a00053fb2f518', 'request_id': 'req-58f01e90-30ab-4468-a73d-c03fa246231e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:21.525370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc815b360e14d08bc0a00053fb2f518', 'request_id': 'req-58f01e90-30ab-4468-a73d-c03fa246231e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:21.551102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc815b360e14d08bc0a00053fb2f518', 'request_id': 'req-58f01e90-30ab-4468-a73d-c03fa246231e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:21.592999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] The token cache is being invalidate because project ffc815b360e14d08bc0a00053fb2f518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:21.593324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:21.594070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:21.596839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1101/2207] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:21 2022] DELETE /identity/v3/projects/ffc815b360e14d08bc0a00053fb2f518 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:21.604392 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:21.606205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:21.641687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38676c8-7805-4892-9317-8d800f956895 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:21.844739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fac6e-c543-40b6-8cbc-1c48028f5fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:21.845531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fac6e-c543-40b6-8cbc-1c48028f5fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:21.845895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fac6e-c543-40b6-8cbc-1c48028f5fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:21.846738 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fac6e-c543-40b6-8cbc-1c48028f5fee admin admin] PATH_INFO: `/v3/users/c2db3de1b0aa425c8d70254a8ca44983` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:21.850159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323fac6e-c543-40b6-8cbc-1c48028f5fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2db3de1b0aa425c8d70254a8ca44983)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:21.852359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323fac6e-c543-40b6-8cbc-1c48028f5fee admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:21.956403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-323fac6e-c543-40b6-8cbc-1c48028f5fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2db3de1b0aa425c8d70254a8ca44983'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:21.970797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-323fac6e-c543-40b6-8cbc-1c48028f5fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2db3de1b0aa425c8d70254a8ca44983'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:22.012077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-323fac6e-c543-40b6-8cbc-1c48028f5fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2db3de1b0aa425c8d70254a8ca44983'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:22.014474 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-323fac6e-c543-40b6-8cbc-1c48028f5fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2db3de1b0aa425c8d70254a8ca44983'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:22.015433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-323fac6e-c543-40b6-8cbc-1c48028f5fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2db3de1b0aa425c8d70254a8ca44983'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:22.047783 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1107/2208] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:21 2022] DELETE /identity/v3/users/c2db3de1b0aa425c8d70254a8ca44983 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:22.719232 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:22.723914 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:22.756062 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f01e90-30ab-4468-a73d-c03fa246231e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:22.897270 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6b390-0d15-4ede-8d16-390630e37c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:22.897999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6b390-0d15-4ede-8d16-390630e37c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:22.898313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6b390-0d15-4ede-8d16-390630e37c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:22.898575 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6b390-0d15-4ede-8d16-390630e37c48 admin admin] PATH_INFO: `/v3/projects/8593f3f2b3364ec68b80be5941f755ba` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:22.899938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6b390-0d15-4ede-8d16-390630e37c48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8593f3f2b3364ec68b80be5941f755ba)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:22.912326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6b390-0d15-4ede-8d16-390630e37c48 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:22.999238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:23.063913 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e1b6b390-0d15-4ede-8d16-390630e37c48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8593f3f2b3364ec68b80be5941f755ba', 'request_id': 'req-e1b6b390-0d15-4ede-8d16-390630e37c48'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:23.081925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e1b6b390-0d15-4ede-8d16-390630e37c48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8593f3f2b3364ec68b80be5941f755ba', 'request_id': 'req-e1b6b390-0d15-4ede-8d16-390630e37c48'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:23.092665 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e1b6b390-0d15-4ede-8d16-390630e37c48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8593f3f2b3364ec68b80be5941f755ba', 'request_id': 'req-e1b6b390-0d15-4ede-8d16-390630e37c48'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:23.139290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e1b6b390-0d15-4ede-8d16-390630e37c48 admin admin] The token cache is being invalidate because project 8593f3f2b3364ec68b80be5941f755ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:23.139843 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e1b6b390-0d15-4ede-8d16-390630e37c48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:23.140815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e1b6b390-0d15-4ede-8d16-390630e37c48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:23.143517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1102/2209] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:22 2022] DELETE /identity/v3/projects/8593f3f2b3364ec68b80be5941f755ba => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:23.150452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a8abe-1829-42b2-8e4f-cf0ac9af3cdb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:23.151234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a8abe-1829-42b2-8e4f-cf0ac9af3cdb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:23.151234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a8abe-1829-42b2-8e4f-cf0ac9af3cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:23.191690 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-678a8abe-1829-42b2-8e4f-cf0ac9af3cdb None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:23.193274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-678a8abe-1829-42b2-8e4f-cf0ac9af3cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:23.220194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:23.353833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1108/2210] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:23.364812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384334f-cf5a-42b3-9674-f20d85e6ddfb None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:23.365412 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384334f-cf5a-42b3-9674-f20d85e6ddfb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:23.365869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384334f-cf5a-42b3-9674-f20d85e6ddfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:23.395512 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0384334f-cf5a-42b3-9674-f20d85e6ddfb None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:23.396752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0384334f-cf5a-42b3-9674-f20d85e6ddfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:23.414372 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1103/2211] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:23.425476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a8abe-1829-42b2-8e4f-cf0ac9af3cdb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:23.436163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a8abe-1829-42b2-8e4f-cf0ac9af3cdb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:23.443896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435d5aa-f6ea-40c9-b052-da0eb293c472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:23.444950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435d5aa-f6ea-40c9-b052-da0eb293c472 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:23.445529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435d5aa-f6ea-40c9-b052-da0eb293c472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:23.446241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435d5aa-f6ea-40c9-b052-da0eb293c472 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:23.447326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435d5aa-f6ea-40c9-b052-da0eb293c472 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:39:23.447880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435d5aa-f6ea-40c9-b052-da0eb293c472 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:23.450469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435d5aa-f6ea-40c9-b052-da0eb293c472 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:23.478756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1109/2212] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:39:23.481950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384334f-cf5a-42b3-9674-f20d85e6ddfb None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:23.497273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384334f-cf5a-42b3-9674-f20d85e6ddfb None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:23.510041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1d36b-e878-4453-b3e3-628a7604ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:23.510903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1d36b-e878-4453-b3e3-628a7604ac6b admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:23.511316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1d36b-e878-4453-b3e3-628a7604ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:23.511674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1d36b-e878-4453-b3e3-628a7604ac6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:23.512722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1d36b-e878-4453-b3e3-628a7604ac6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:23.514823 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1d36b-e878-4453-b3e3-628a7604ac6b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:23.581116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1104/2213] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:23 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:23.585337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435d5aa-f6ea-40c9-b052-da0eb293c472 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:23.599096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435d5aa-f6ea-40c9-b052-da0eb293c472 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:23.752836 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b35490-527b-4e10-80d3-8c2d7bec5051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:23.753887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b35490-527b-4e10-80d3-8c2d7bec5051 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:23.754492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b35490-527b-4e10-80d3-8c2d7bec5051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:23.754818 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b35490-527b-4e10-80d3-8c2d7bec5051 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:23.755482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b35490-527b-4e10-80d3-8c2d7bec5051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:23.762505 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b35490-527b-4e10-80d3-8c2d7bec5051 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:23.815504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1110/2214] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:23 2022] POST /identity/v3/users => generated 285 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:23.819734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1d36b-e878-4453-b3e3-628a7604ac6b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:23.835051 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1d36b-e878-4453-b3e3-628a7604ac6b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:23.840894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1fccf-b625-4bed-8d28-13424d066a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:23.841799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1fccf-b625-4bed-8d28-13424d066a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:23.842248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1fccf-b625-4bed-8d28-13424d066a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:23.842679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1fccf-b625-4bed-8d28-13424d066a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:23.843619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1fccf-b625-4bed-8d28-13424d066a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:23.847604 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1fccf-b625-4bed-8d28-13424d066a62 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:23.863259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1105/2215] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:23.866862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b35490-527b-4e10-80d3-8c2d7bec5051 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:23.889330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b35490-527b-4e10-80d3-8c2d7bec5051 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:23.895239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a52e95-f3b0-4fe4-a560-2aa95fa92466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:23.896199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a52e95-f3b0-4fe4-a560-2aa95fa92466 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:23.896672 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a52e95-f3b0-4fe4-a560-2aa95fa92466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:23.897236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a52e95-f3b0-4fe4-a560-2aa95fa92466 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:23.898840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a52e95-f3b0-4fe4-a560-2aa95fa92466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:23.900903 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a52e95-f3b0-4fe4-a560-2aa95fa92466 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:23.925753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1111/2216] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:39:23.931690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1fccf-b625-4bed-8d28-13424d066a62 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:23.974193 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1fccf-b625-4bed-8d28-13424d066a62 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:23.988490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33c5dd-d8cb-4aa1-add5-131933b63789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:23.990246 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33c5dd-d8cb-4aa1-add5-131933b63789 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:23.993317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33c5dd-d8cb-4aa1-add5-131933b63789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:23.994343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33c5dd-d8cb-4aa1-add5-131933b63789 admin admin] PATH_INFO: `/v3/projects/5ffec2e24e134e6188ab7103be544972/users/ff7e97e785a54bf7845179fae2a6badb/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:24.030620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33c5dd-d8cb-4aa1-add5-131933b63789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ffec2e24e134e6188ab7103be544972, user_id=ff7e97e785a54bf7845179fae2a6badb, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:24.034771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33c5dd-d8cb-4aa1-add5-131933b63789 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:24.077232 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1106/2217] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:23 2022] PUT /identity/v3/projects/5ffec2e24e134e6188ab7103be544972/users/ff7e97e785a54bf7845179fae2a6badb/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:24.094741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4617ce5-368c-4068-ac8b-5f420fa8e3a1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:24.095125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4617ce5-368c-4068-ac8b-5f420fa8e3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:24.095428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4617ce5-368c-4068-ac8b-5f420fa8e3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:24.165465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b4617ce5-368c-4068-ac8b-5f420fa8e3a1 None None] MFA Rules not processed for user `ff7e97e785a54bf7845179fae2a6badb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:24.168139 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b4617ce5-368c-4068-ac8b-5f420fa8e3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:24.283732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1112/2218] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:24.296985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33c5dd-d8cb-4aa1-add5-131933b63789 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:24.299835 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9e33c5dd-d8cb-4aa1-add5-131933b63789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:24.397268 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33c5dd-d8cb-4aa1-add5-131933b63789 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:24.412587 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d170b4-d4ee-4953-b3f5-14c9ce11d59f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f57fa108c3d478a9976f4aea5f33bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:24.414013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d170b4-d4ee-4953-b3f5-14c9ce11d59f service glance] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:24.414623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d170b4-d4ee-4953-b3f5-14c9ce11d59f service glance] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:24.415051 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d170b4-d4ee-4953-b3f5-14c9ce11d59f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:24.460405 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d170b4-d4ee-4953-b3f5-14c9ce11d59f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:24.465582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d170b4-d4ee-4953-b3f5-14c9ce11d59f service glance] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:24.490161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1107/2219] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:24 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:29.161511 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4617ce5-368c-4068-ac8b-5f420fa8e3a1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:29.164146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b4617ce5-368c-4068-ac8b-5f420fa8e3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:29.183985 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:29.292265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4617ce5-368c-4068-ac8b-5f420fa8e3a1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:29.301258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:29.301998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:29.302567 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:29.303080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] PATH_INFO: `/v3/users/7d52bd0da53c4423be51f3b179ac20f9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:29.304287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d52bd0da53c4423be51f3b179ac20f9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:29.306339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:29.406984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d52bd0da53c4423be51f3b179ac20f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:29.425055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d52bd0da53c4423be51f3b179ac20f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:29.446802 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d52bd0da53c4423be51f3b179ac20f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:29.447650 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d52bd0da53c4423be51f3b179ac20f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:29.448775 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d52bd0da53c4423be51f3b179ac20f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:29.523713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1113/2220] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:29 2022] DELETE /identity/v3/users/7d52bd0da53c4423be51f3b179ac20f9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:29.529123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8964b-1df3-4cf4-a09a-c4349b4153ba None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:29.529693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8964b-1df3-4cf4-a09a-c4349b4153ba None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:29.530012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8964b-1df3-4cf4-a09a-c4349b4153ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:29.558053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b7d8964b-1df3-4cf4-a09a-c4349b4153ba None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:29.560101 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b7d8964b-1df3-4cf4-a09a-c4349b4153ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:29.649298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:29.676053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1108/2221] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:29.681046 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:29.685422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:29.777438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311cfb8-9a10-44da-9b9e-fe4826bb9744 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:29.782484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd524eb-8d20-4690-afc2-1930de874175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:29.783628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd524eb-8d20-4690-afc2-1930de874175 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:29.783910 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd524eb-8d20-4690-afc2-1930de874175 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:29.784186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd524eb-8d20-4690-afc2-1930de874175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:29.800767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd524eb-8d20-4690-afc2-1930de874175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:29.802648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd524eb-8d20-4690-afc2-1930de874175 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:29.817181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1114/2222] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:30.156267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8964b-1df3-4cf4-a09a-c4349b4153ba None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:30.158328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b7d8964b-1df3-4cf4-a09a-c4349b4153ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:30.185304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8964b-1df3-4cf4-a09a-c4349b4153ba None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:30.194544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2045db5-c477-4dc7-b1e5-1ab730a7a740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:30.195855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2045db5-c477-4dc7-b1e5-1ab730a7a740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:30.196281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2045db5-c477-4dc7-b1e5-1ab730a7a740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:30.196659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2045db5-c477-4dc7-b1e5-1ab730a7a740 admin admin] PATH_INFO: `/v3/projects/0b13f6a12091447da7be7bde4169a139` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:30.198710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2045db5-c477-4dc7-b1e5-1ab730a7a740 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b13f6a12091447da7be7bde4169a139)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:30.202073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2045db5-c477-4dc7-b1e5-1ab730a7a740 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:30.281466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e2045db5-c477-4dc7-b1e5-1ab730a7a740 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b13f6a12091447da7be7bde4169a139', 'request_id': 'req-e2045db5-c477-4dc7-b1e5-1ab730a7a740'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:30.290582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e2045db5-c477-4dc7-b1e5-1ab730a7a740 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b13f6a12091447da7be7bde4169a139', 'request_id': 'req-e2045db5-c477-4dc7-b1e5-1ab730a7a740'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:30.299869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e2045db5-c477-4dc7-b1e5-1ab730a7a740 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b13f6a12091447da7be7bde4169a139', 'request_id': 'req-e2045db5-c477-4dc7-b1e5-1ab730a7a740'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:30.316381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e2045db5-c477-4dc7-b1e5-1ab730a7a740 admin admin] The token cache is being invalidate because project 0b13f6a12091447da7be7bde4169a139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:30.316715 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e2045db5-c477-4dc7-b1e5-1ab730a7a740 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:30.321893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e2045db5-c477-4dc7-b1e5-1ab730a7a740 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:30.327885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1109/2223] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:30 2022] DELETE /identity/v3/projects/0b13f6a12091447da7be7bde4169a139 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:30.332613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb57c7-6cd1-42e7-9c80-92ab0aae3739 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:30.333109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb57c7-6cd1-42e7-9c80-92ab0aae3739 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:30.333472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb57c7-6cd1-42e7-9c80-92ab0aae3739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:30.372052 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1dbb57c7-6cd1-42e7-9c80-92ab0aae3739 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:30.373396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1dbb57c7-6cd1-42e7-9c80-92ab0aae3739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:30.421582 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:30.482785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1115/2224] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:30.499170 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063cef4-03da-4f60-a0b8-c21e5cc992f1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:30.499170 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063cef4-03da-4f60-a0b8-c21e5cc992f1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:30.499170 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063cef4-03da-4f60-a0b8-c21e5cc992f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:30.523281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5063cef4-03da-4f60-a0b8-c21e5cc992f1 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:30.525355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5063cef4-03da-4f60-a0b8-c21e5cc992f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:30.536537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1110/2225] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:39:30.541347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb57c7-6cd1-42e7-9c80-92ab0aae3739 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:30.556152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb57c7-6cd1-42e7-9c80-92ab0aae3739 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:30.561409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c3aa2-80d9-43a6-a30d-47afe0ccab92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:30.562106 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c3aa2-80d9-43a6-a30d-47afe0ccab92 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:30.562502 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c3aa2-80d9-43a6-a30d-47afe0ccab92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:30.562863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c3aa2-80d9-43a6-a30d-47afe0ccab92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:30.563703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c3aa2-80d9-43a6-a30d-47afe0ccab92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:39:30.564675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c3aa2-80d9-43a6-a30d-47afe0ccab92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:30.566288 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c3aa2-80d9-43a6-a30d-47afe0ccab92 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:30.583340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1116/2226] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:39:30.584423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063cef4-03da-4f60-a0b8-c21e5cc992f1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:30.603348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063cef4-03da-4f60-a0b8-c21e5cc992f1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:30.625805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011769b-583b-487f-87e9-f2761d3782f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:30.627371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011769b-583b-487f-87e9-f2761d3782f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:30.627682 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011769b-583b-487f-87e9-f2761d3782f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:30.628086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011769b-583b-487f-87e9-f2761d3782f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:30.628834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011769b-583b-487f-87e9-f2761d3782f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:30.631816 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011769b-583b-487f-87e9-f2761d3782f9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:30.686818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1111/2227] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:30 2022] POST /identity/v3/projects => generated 397 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:30.691329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c3aa2-80d9-43a6-a30d-47afe0ccab92 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:30.701980 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c3aa2-80d9-43a6-a30d-47afe0ccab92 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:30.806017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc428e-4e1f-4fd1-8de9-19f78f53df97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:30.806739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc428e-4e1f-4fd1-8de9-19f78f53df97 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:30.807039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc428e-4e1f-4fd1-8de9-19f78f53df97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:30.807377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc428e-4e1f-4fd1-8de9-19f78f53df97 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:30.808221 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc428e-4e1f-4fd1-8de9-19f78f53df97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:30.810363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc428e-4e1f-4fd1-8de9-19f78f53df97 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:30.865316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1117/2228] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:30 2022] POST /identity/v3/users => generated 299 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:30.865690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011769b-583b-487f-87e9-f2761d3782f9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:30.884248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011769b-583b-487f-87e9-f2761d3782f9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:30.897081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e242374-157a-490a-a365-e75b7296b655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:30.897881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e242374-157a-490a-a365-e75b7296b655 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:30.898269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e242374-157a-490a-a365-e75b7296b655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:30.898537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e242374-157a-490a-a365-e75b7296b655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:30.899731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e242374-157a-490a-a365-e75b7296b655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:30.902056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e242374-157a-490a-a365-e75b7296b655 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:30.926417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1112/2229] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:39:30.927738 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc428e-4e1f-4fd1-8de9-19f78f53df97 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:30.939699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc428e-4e1f-4fd1-8de9-19f78f53df97 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:30.945893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0a4ea-43f7-4096-a798-d0eb05b80a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:30.946765 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0a4ea-43f7-4096-a798-d0eb05b80a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:30.947268 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0a4ea-43f7-4096-a798-d0eb05b80a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:30.947706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0a4ea-43f7-4096-a798-d0eb05b80a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:30.948679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0a4ea-43f7-4096-a798-d0eb05b80a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:30.950530 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0a4ea-43f7-4096-a798-d0eb05b80a99 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:30.972194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1118/2230] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:30.972365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e242374-157a-490a-a365-e75b7296b655 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:30.986551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e242374-157a-490a-a365-e75b7296b655 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:30.991751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a910b7c-c349-48ef-b4f7-a443362308c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:30.992648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a910b7c-c349-48ef-b4f7-a443362308c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:30.993102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a910b7c-c349-48ef-b4f7-a443362308c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:30.993561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a910b7c-c349-48ef-b4f7-a443362308c5 admin admin] PATH_INFO: `/v3/projects/80c17e36a61f4d3e9d22c96583d2d8db/users/6cde644bf3eb4ee98a00516402a507f1/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:31.010875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a910b7c-c349-48ef-b4f7-a443362308c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c17e36a61f4d3e9d22c96583d2d8db, user_id=6cde644bf3eb4ee98a00516402a507f1, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:31.014475 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a910b7c-c349-48ef-b4f7-a443362308c5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:31.033545 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1113/2231] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:30 2022] PUT /identity/v3/projects/80c17e36a61f4d3e9d22c96583d2d8db/users/6cde644bf3eb4ee98a00516402a507f1/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:31.046847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197492e-7422-4f5e-96a3-dc9cd8a24258 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:31.047124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197492e-7422-4f5e-96a3-dc9cd8a24258 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:31.047236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197492e-7422-4f5e-96a3-dc9cd8a24258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:31.082811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f197492e-7422-4f5e-96a3-dc9cd8a24258 None None] MFA Rules not processed for user `6cde644bf3eb4ee98a00516402a507f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:31.084395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f197492e-7422-4f5e-96a3-dc9cd8a24258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:31.169179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1119/2232] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:31.185273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a910b7c-c349-48ef-b4f7-a443362308c5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:31.187118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8a910b7c-c349-48ef-b4f7-a443362308c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:31.260332 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a910b7c-c349-48ef-b4f7-a443362308c5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:31.264992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b422a2-3ba5-465f-93fd-db5e0726ff35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:31.265997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b422a2-3ba5-465f-93fd-db5e0726ff35 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:31.266301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b422a2-3ba5-465f-93fd-db5e0726ff35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:31.266576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b422a2-3ba5-465f-93fd-db5e0726ff35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:31.278152 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b422a2-3ba5-465f-93fd-db5e0726ff35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:31.279713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b422a2-3ba5-465f-93fd-db5e0726ff35 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:31.291393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1114/2233] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:31.656001 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197492e-7422-4f5e-96a3-dc9cd8a24258 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:31.658429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f197492e-7422-4f5e-96a3-dc9cd8a24258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:31.741689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:31.940002 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197492e-7422-4f5e-96a3-dc9cd8a24258 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:31.958636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:31.959516 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:31.959871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:31.960748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] PATH_INFO: `/v3/users/30f8f1c689544a4e87bb2080b5bd4759` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:31.966434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30f8f1c689544a4e87bb2080b5bd4759)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:31.974317 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:32.066877 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30f8f1c689544a4e87bb2080b5bd4759'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:32.084257 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30f8f1c689544a4e87bb2080b5bd4759'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:32.108045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30f8f1c689544a4e87bb2080b5bd4759'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:32.108913 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30f8f1c689544a4e87bb2080b5bd4759'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:32.110330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30f8f1c689544a4e87bb2080b5bd4759'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:32.132240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1120/2234] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:31 2022] DELETE /identity/v3/users/30f8f1c689544a4e87bb2080b5bd4759 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:32.134028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35da3f0-fee1-4539-913c-d77cc51437e6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:32.134503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35da3f0-fee1-4539-913c-d77cc51437e6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:32.134678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35da3f0-fee1-4539-913c-d77cc51437e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:32.178930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d35da3f0-fee1-4539-913c-d77cc51437e6 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:32.183874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d35da3f0-fee1-4539-913c-d77cc51437e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:32.192816 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:32.292250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1115/2235] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:32.303493 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:32.306372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:32.388632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b316481-5b6b-4832-b21d-3a2e5c01bba1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:32.393990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4b1b8-04d7-4c8d-85ad-ca31c3867f0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:32.394780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4b1b8-04d7-4c8d-85ad-ca31c3867f0a service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:32.395112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4b1b8-04d7-4c8d-85ad-ca31c3867f0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:32.395420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4b1b8-04d7-4c8d-85ad-ca31c3867f0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:32.406704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4b1b8-04d7-4c8d-85ad-ca31c3867f0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:32.409057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4b1b8-04d7-4c8d-85ad-ca31c3867f0a service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:32.420301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1121/2236] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:32.781047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35da3f0-fee1-4539-913c-d77cc51437e6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:32.784335 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d35da3f0-fee1-4539-913c-d77cc51437e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:32.813750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35da3f0-fee1-4539-913c-d77cc51437e6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:32.824635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819a860-302f-45e6-ba04-c8e038b8367f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:32.825398 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819a860-302f-45e6-ba04-c8e038b8367f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:32.825941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819a860-302f-45e6-ba04-c8e038b8367f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:32.826325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819a860-302f-45e6-ba04-c8e038b8367f admin admin] PATH_INFO: `/v3/projects/1f2047e7c9724b34bde99e63c07d7e8a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:32.828080 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819a860-302f-45e6-ba04-c8e038b8367f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f2047e7c9724b34bde99e63c07d7e8a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:32.830921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819a860-302f-45e6-ba04-c8e038b8367f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:32.971785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c819a860-302f-45e6-ba04-c8e038b8367f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2047e7c9724b34bde99e63c07d7e8a', 'request_id': 'req-c819a860-302f-45e6-ba04-c8e038b8367f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:32.985962 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c819a860-302f-45e6-ba04-c8e038b8367f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2047e7c9724b34bde99e63c07d7e8a', 'request_id': 'req-c819a860-302f-45e6-ba04-c8e038b8367f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:32.999481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c819a860-302f-45e6-ba04-c8e038b8367f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2047e7c9724b34bde99e63c07d7e8a', 'request_id': 'req-c819a860-302f-45e6-ba04-c8e038b8367f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:33.019757 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c819a860-302f-45e6-ba04-c8e038b8367f admin admin] The token cache is being invalidate because project 1f2047e7c9724b34bde99e63c07d7e8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:33.020057 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c819a860-302f-45e6-ba04-c8e038b8367f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:33.020689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c819a860-302f-45e6-ba04-c8e038b8367f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:33.022244 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1116/2237] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:32 2022] DELETE /identity/v3/projects/1f2047e7c9724b34bde99e63c07d7e8a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:39:33.028312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a8b41-0078-428c-b702-231129ae19b8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:33.029673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a8b41-0078-428c-b702-231129ae19b8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:33.030164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a8b41-0078-428c-b702-231129ae19b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:33.066848 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b01a8b41-0078-428c-b702-231129ae19b8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:33.068256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b01a8b41-0078-428c-b702-231129ae19b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:33.147420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:33.211103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1122/2238] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:33.219052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9b0d3-f1a1-4aa1-931a-b54684b58d2c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:33.219622 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9b0d3-f1a1-4aa1-931a-b54684b58d2c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:33.219730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9b0d3-f1a1-4aa1-931a-b54684b58d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:33.265806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f7a9b0d3-f1a1-4aa1-931a-b54684b58d2c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:33.273465 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f7a9b0d3-f1a1-4aa1-931a-b54684b58d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:33.302790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1117/2239] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:33.307658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a8b41-0078-428c-b702-231129ae19b8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:33.323570 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a8b41-0078-428c-b702-231129ae19b8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:33.338718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14290a-919c-483f-ad73-7b3e13754496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:33.340601 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14290a-919c-483f-ad73-7b3e13754496 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:33.340969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14290a-919c-483f-ad73-7b3e13754496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:33.342131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14290a-919c-483f-ad73-7b3e13754496 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:33.343006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14290a-919c-483f-ad73-7b3e13754496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:39:33.345398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14290a-919c-483f-ad73-7b3e13754496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:33.347516 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14290a-919c-483f-ad73-7b3e13754496 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:33.431386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1123/2240] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:39:33.435228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9b0d3-f1a1-4aa1-931a-b54684b58d2c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:33.449144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9b0d3-f1a1-4aa1-931a-b54684b58d2c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:33.455654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb1999-0ce9-4086-af5b-be37975894df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:33.456577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb1999-0ce9-4086-af5b-be37975894df admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:33.457216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb1999-0ce9-4086-af5b-be37975894df admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:33.457719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb1999-0ce9-4086-af5b-be37975894df admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:33.459391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb1999-0ce9-4086-af5b-be37975894df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:33.461733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb1999-0ce9-4086-af5b-be37975894df admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:33.499732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1118/2241] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:33 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:33.506094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14290a-919c-483f-ad73-7b3e13754496 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:33.525870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14290a-919c-483f-ad73-7b3e13754496 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:33.607771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88964188-5163-4002-b92c-f98cea7b088b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:33.608796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88964188-5163-4002-b92c-f98cea7b088b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:33.609158 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88964188-5163-4002-b92c-f98cea7b088b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:33.609581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88964188-5163-4002-b92c-f98cea7b088b admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:33.610259 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88964188-5163-4002-b92c-f98cea7b088b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:33.612823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88964188-5163-4002-b92c-f98cea7b088b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:33.643692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1124/2242] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:33 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:39:33.644961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb1999-0ce9-4086-af5b-be37975894df admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:33.670724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb1999-0ce9-4086-af5b-be37975894df admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:33.678542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a0cd8-e560-4e59-8820-16aa8a4231e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:33.679934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a0cd8-e560-4e59-8820-16aa8a4231e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:33.680399 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a0cd8-e560-4e59-8820-16aa8a4231e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:33.680923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a0cd8-e560-4e59-8820-16aa8a4231e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:33.682456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a0cd8-e560-4e59-8820-16aa8a4231e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:33.684837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a0cd8-e560-4e59-8820-16aa8a4231e5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:33.699466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1119/2243] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:33.703419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88964188-5163-4002-b92c-f98cea7b088b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:33.724867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88964188-5163-4002-b92c-f98cea7b088b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:33.731382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb545d0-9a25-4a67-9272-7ec89391c42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:33.732069 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb545d0-9a25-4a67-9272-7ec89391c42d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:33.732385 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb545d0-9a25-4a67-9272-7ec89391c42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:33.732662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb545d0-9a25-4a67-9272-7ec89391c42d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:33.733427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb545d0-9a25-4a67-9272-7ec89391c42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:33.735285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb545d0-9a25-4a67-9272-7ec89391c42d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:33.753757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1125/2244] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:39:33.758507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a0cd8-e560-4e59-8820-16aa8a4231e5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:33.770659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a0cd8-e560-4e59-8820-16aa8a4231e5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:33.775446 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482f91f-644e-4c32-acb6-111123fdee87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:33.776141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482f91f-644e-4c32-acb6-111123fdee87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:33.776424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482f91f-644e-4c32-acb6-111123fdee87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:33.776705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482f91f-644e-4c32-acb6-111123fdee87 admin admin] PATH_INFO: `/v3/projects/810069989cfb42799139be8ec2423288/users/890be2e19b43473caa79b2dc889f8436/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:33.793041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482f91f-644e-4c32-acb6-111123fdee87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=810069989cfb42799139be8ec2423288, user_id=890be2e19b43473caa79b2dc889f8436, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:33.795819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482f91f-644e-4c32-acb6-111123fdee87 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:33.812858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1120/2245] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:33 2022] PUT /identity/v3/projects/810069989cfb42799139be8ec2423288/users/890be2e19b43473caa79b2dc889f8436/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:39:33.819362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f2552-edad-49dd-98b1-9d0d976320ab None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:33.819737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f2552-edad-49dd-98b1-9d0d976320ab None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:33.820024 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f2552-edad-49dd-98b1-9d0d976320ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:33.841032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-425f2552-edad-49dd-98b1-9d0d976320ab None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:33.842439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-425f2552-edad-49dd-98b1-9d0d976320ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:33.908177 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1126/2246] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:33.921454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482f91f-644e-4c32-acb6-111123fdee87 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:33.924217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7482f91f-644e-4c32-acb6-111123fdee87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:33.963016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:33.993020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482f91f-644e-4c32-acb6-111123fdee87 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:33.998425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d140ff-aa85-41f8-933e-69d1356e6a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:33.999286 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d140ff-aa85-41f8-933e-69d1356e6a43 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:33.999863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d140ff-aa85-41f8-933e-69d1356e6a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:34.000319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d140ff-aa85-41f8-933e-69d1356e6a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:34.012815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d140ff-aa85-41f8-933e-69d1356e6a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:34.014963 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d140ff-aa85-41f8-933e-69d1356e6a43 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:34.027827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1121/2247] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:41.449810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f2552-edad-49dd-98b1-9d0d976320ab None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:41.451381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-425f2552-edad-49dd-98b1-9d0d976320ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:41.475982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f2552-edad-49dd-98b1-9d0d976320ab None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:41.494359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd580d-7c34-4d7b-bc76-753b30db38df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:41.495030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd580d-7c34-4d7b-bc76-753b30db38df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:41.495336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd580d-7c34-4d7b-bc76-753b30db38df admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:41.495614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd580d-7c34-4d7b-bc76-753b30db38df admin admin] PATH_INFO: `/v3/users/6cde644bf3eb4ee98a00516402a507f1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:41.498832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd580d-7c34-4d7b-bc76-753b30db38df admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cde644bf3eb4ee98a00516402a507f1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:41.503622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd580d-7c34-4d7b-bc76-753b30db38df admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:41.647267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c5cd580d-7c34-4d7b-bc76-753b30db38df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cde644bf3eb4ee98a00516402a507f1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:41.651855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:41.661844 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c5cd580d-7c34-4d7b-bc76-753b30db38df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cde644bf3eb4ee98a00516402a507f1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:41.703159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c5cd580d-7c34-4d7b-bc76-753b30db38df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cde644bf3eb4ee98a00516402a507f1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:41.704962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c5cd580d-7c34-4d7b-bc76-753b30db38df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cde644bf3eb4ee98a00516402a507f1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:41.705996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c5cd580d-7c34-4d7b-bc76-753b30db38df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cde644bf3eb4ee98a00516402a507f1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:41.730285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1127/2248] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:41 2022] DELETE /identity/v3/users/6cde644bf3eb4ee98a00516402a507f1 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:41.736300 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9bd33-7eba-41c2-9e9c-c9a779bf8347 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:41.737515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9bd33-7eba-41c2-9e9c-c9a779bf8347 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:41.738071 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9bd33-7eba-41c2-9e9c-c9a779bf8347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:41.768203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ada9bd33-7eba-41c2-9e9c-c9a779bf8347 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:41.770065 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ada9bd33-7eba-41c2-9e9c-c9a779bf8347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:41.814709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0803195-fe27-4c8e-9abd-89dca5ed4d76 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:41.815607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0803195-fe27-4c8e-9abd-89dca5ed4d76 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:41.816504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0803195-fe27-4c8e-9abd-89dca5ed4d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:41.945535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a0803195-fe27-4c8e-9abd-89dca5ed4d76 None None] MFA Rules not processed for user `890be2e19b43473caa79b2dc889f8436`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:41.951774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a0803195-fe27-4c8e-9abd-89dca5ed4d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:42.040784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1122/2249] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:42.054248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9bd33-7eba-41c2-9e9c-c9a779bf8347 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:42.060260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ada9bd33-7eba-41c2-9e9c-c9a779bf8347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:42.165852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1128/2250] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:42.174422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0803195-fe27-4c8e-9abd-89dca5ed4d76 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:42.184368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a0803195-fe27-4c8e-9abd-89dca5ed4d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:42.228845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0803195-fe27-4c8e-9abd-89dca5ed4d76 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:42.253130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef241e88-9143-4276-a10f-dc360547f28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:42.254979 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef241e88-9143-4276-a10f-dc360547f28c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:42.255329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef241e88-9143-4276-a10f-dc360547f28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:42.255616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef241e88-9143-4276-a10f-dc360547f28c admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:42.256301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef241e88-9143-4276-a10f-dc360547f28c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:42.259880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef241e88-9143-4276-a10f-dc360547f28c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:42.267466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9bd33-7eba-41c2-9e9c-c9a779bf8347 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:42.272583 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906538ff-d2b4-476d-8ee4-60f98d0855c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:42.275507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906538ff-d2b4-476d-8ee4-60f98d0855c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:42.283623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906538ff-d2b4-476d-8ee4-60f98d0855c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:42.287644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906538ff-d2b4-476d-8ee4-60f98d0855c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:42.328646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1129/2251] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:42 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:42.337566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef241e88-9143-4276-a10f-dc360547f28c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:42.371359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906538ff-d2b4-476d-8ee4-60f98d0855c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:42.376612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906538ff-d2b4-476d-8ee4-60f98d0855c8 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:42.398987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1123/2252] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:42.524407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef241e88-9143-4276-a10f-dc360547f28c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:42.540500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312ef6a-fab8-4594-bfb3-8697d848d1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:42.541153 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312ef6a-fab8-4594-bfb3-8697d848d1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:42.541435 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312ef6a-fab8-4594-bfb3-8697d848d1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:42.541709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312ef6a-fab8-4594-bfb3-8697d848d1d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:42.542529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312ef6a-fab8-4594-bfb3-8697d848d1d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:42.545996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312ef6a-fab8-4594-bfb3-8697d848d1d4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:42.602496 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906538ff-d2b4-476d-8ee4-60f98d0855c8 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:42.603144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1130/2253] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:42 2022] POST /identity/v3/users => generated 295 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:42.614814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906538ff-d2b4-476d-8ee4-60f98d0855c8 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:42.624004 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf334ec0-082a-4de7-ae30-b7a3008a2782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:42.624355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf334ec0-082a-4de7-ae30-b7a3008a2782 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:42.624730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf334ec0-082a-4de7-ae30-b7a3008a2782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:42.624905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf334ec0-082a-4de7-ae30-b7a3008a2782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:42.625603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf334ec0-082a-4de7-ae30-b7a3008a2782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:42.636404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf334ec0-082a-4de7-ae30-b7a3008a2782 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:42.696476 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1124/2254] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:42.701336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312ef6a-fab8-4594-bfb3-8697d848d1d4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:42.725145 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312ef6a-fab8-4594-bfb3-8697d848d1d4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:42.750402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f62ec6-0423-45e9-a8aa-9d6042d4cd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:42.752261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f62ec6-0423-45e9-a8aa-9d6042d4cd4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:42.753352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f62ec6-0423-45e9-a8aa-9d6042d4cd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:42.753653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f62ec6-0423-45e9-a8aa-9d6042d4cd4d admin admin] PATH_INFO: `/v3/projects/ec8569655b2542939019e71ea94d1ddd/users/5d7cf9c678b44bc9876e2c1efaa7277d/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:42.827035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f62ec6-0423-45e9-a8aa-9d6042d4cd4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8569655b2542939019e71ea94d1ddd, user_id=5d7cf9c678b44bc9876e2c1efaa7277d, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:42.830700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f62ec6-0423-45e9-a8aa-9d6042d4cd4d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:42.851023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:42.869450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1131/2255] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:42 2022] PUT /identity/v3/projects/ec8569655b2542939019e71ea94d1ddd/users/5d7cf9c678b44bc9876e2c1efaa7277d/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:42.873699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf334ec0-082a-4de7-ae30-b7a3008a2782 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:42.906541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf334ec0-082a-4de7-ae30-b7a3008a2782 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:43.081183 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad22f02-2641-46df-b0a7-07e9d18dbb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:43.084050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad22f02-2641-46df-b0a7-07e9d18dbb5d admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:43.084405 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad22f02-2641-46df-b0a7-07e9d18dbb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:43.084690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad22f02-2641-46df-b0a7-07e9d18dbb5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:43.088637 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad22f02-2641-46df-b0a7-07e9d18dbb5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:43.092632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad22f02-2641-46df-b0a7-07e9d18dbb5d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:43.121439 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1125/2256] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:43.126299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f62ec6-0423-45e9-a8aa-9d6042d4cd4d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:43.145175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f62ec6-0423-45e9-a8aa-9d6042d4cd4d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:43.158476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9c416-872a-441f-b6e3-b8456450d488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:43.163961 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9c416-872a-441f-b6e3-b8456450d488 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:43.164326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9c416-872a-441f-b6e3-b8456450d488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:43.165257 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9c416-872a-441f-b6e3-b8456450d488 admin admin] PATH_INFO: `/v3/projects/ec8569655b2542939019e71ea94d1ddd/users/5d7cf9c678b44bc9876e2c1efaa7277d/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:43.174741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9c416-872a-441f-b6e3-b8456450d488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8569655b2542939019e71ea94d1ddd, user_id=5d7cf9c678b44bc9876e2c1efaa7277d, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:43.189236 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f41c4-44a2-470b-b091-422e6d34cdca None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:43.189619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f41c4-44a2-470b-b091-422e6d34cdca None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:43.189947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f41c4-44a2-470b-b091-422e6d34cdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:43.199435 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:43.201534 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9c416-872a-441f-b6e3-b8456450d488 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:43.251595 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-2c3f41c4-44a2-470b-b091-422e6d34cdca None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:43.257405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1132/2257] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:43 2022] PUT /identity/v3/projects/ec8569655b2542939019e71ea94d1ddd/users/5d7cf9c678b44bc9876e2c1efaa7277d/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:39:43.257616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2c3f41c4-44a2-470b-b091-422e6d34cdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:43.392236 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1126/2258] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:39:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:39:43.405115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9c416-872a-441f-b6e3-b8456450d488 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:43.407361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4da9c416-872a-441f-b6e3-b8456450d488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:43.684964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9c416-872a-441f-b6e3-b8456450d488 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:43.739031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f41c4-44a2-470b-b091-422e6d34cdca None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:43.741000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2c3f41c4-44a2-470b-b091-422e6d34cdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:43.742252 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7e486-1bca-4ad0-8f3c-1ee58079207e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:43.744788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e486-1bca-4ad0-8f3c-1ee58079207e service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:43.746568 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e486-1bca-4ad0-8f3c-1ee58079207e service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:43.746867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e486-1bca-4ad0-8f3c-1ee58079207e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:43.794217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7e486-1bca-4ad0-8f3c-1ee58079207e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:43.801105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7e486-1bca-4ad0-8f3c-1ee58079207e service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:43.851050 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1133/2259] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:39:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:39:43.885382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f41c4-44a2-470b-b091-422e6d34cdca None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:43.900176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:43.901001 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:43.901389 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:43.901774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] PATH_INFO: `/v3/projects/80c17e36a61f4d3e9d22c96583d2d8db` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:43.903741 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80c17e36a61f4d3e9d22c96583d2d8db)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:43.905898 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:44.089976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80c17e36a61f4d3e9d22c96583d2d8db', 'request_id': 'req-cca1d551-526d-46ba-8128-9bd5da157856'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:44.121736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80c17e36a61f4d3e9d22c96583d2d8db', 'request_id': 'req-cca1d551-526d-46ba-8128-9bd5da157856'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:44.147926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80c17e36a61f4d3e9d22c96583d2d8db', 'request_id': 'req-cca1d551-526d-46ba-8128-9bd5da157856'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:44.219003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] The token cache is being invalidate because project 80c17e36a61f4d3e9d22c96583d2d8db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:44.220108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:44.222839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:44.251502 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1127/2260] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:43 2022] DELETE /identity/v3/projects/80c17e36a61f4d3e9d22c96583d2d8db => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:47.513236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99213282-774c-4927-a037-7ea91424b66b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:47.514180 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99213282-774c-4927-a037-7ea91424b66b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:47.514610 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99213282-774c-4927-a037-7ea91424b66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:47.580781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-99213282-774c-4927-a037-7ea91424b66b None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:47.582097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-99213282-774c-4927-a037-7ea91424b66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:47.777806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1134/2261] 10.222.0.40 () {58 vars in 967 bytes} [Mon Aug 1 07:39:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:47.792402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:47.793939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:47.983687 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:48.074286 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1d551-526d-46ba-8128-9bd5da157856 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:48.093444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d3f8a-af53-438c-a17e-c161b587c2bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:48.094669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d3f8a-af53-438c-a17e-c161b587c2bc service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:48.094975 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d3f8a-af53-438c-a17e-c161b587c2bc service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:48.095303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d3f8a-af53-438c-a17e-c161b587c2bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:48.120914 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d3f8a-af53-438c-a17e-c161b587c2bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:48.122578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d3f8a-af53-438c-a17e-c161b587c2bc service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:48.153661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1128/2262] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:39:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:39:50.026943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99213282-774c-4927-a037-7ea91424b66b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:50.028748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-99213282-774c-4927-a037-7ea91424b66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:50.167875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99213282-774c-4927-a037-7ea91424b66b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:50.180981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea990ca-a2d8-43da-9460-8b082ae651bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:50.181958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea990ca-a2d8-43da-9460-8b082ae651bb service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:50.182543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea990ca-a2d8-43da-9460-8b082ae651bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:50.183026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea990ca-a2d8-43da-9460-8b082ae651bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:50.185749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2ea990ca-a2d8-43da-9460-8b082ae651bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:50.270627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:50.333228 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea990ca-a2d8-43da-9460-8b082ae651bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:50.335838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea990ca-a2d8-43da-9460-8b082ae651bb service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:50.354467 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1135/2263] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:50 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:51.956052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d3f8a-af53-438c-a17e-c161b587c2bc service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:51.957795 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6b0d3f8a-af53-438c-a17e-c161b587c2bc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:52.103702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d3f8a-af53-438c-a17e-c161b587c2bc service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:52.116006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa7439-2c9c-4ce0-9986-631b7971447f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:52.116811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa7439-2c9c-4ce0-9986-631b7971447f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:52.117253 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa7439-2c9c-4ce0-9986-631b7971447f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:52.117647 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa7439-2c9c-4ce0-9986-631b7971447f admin admin] PATH_INFO: `/v3/users/9255edea8b504727a587443e124f3e7a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:52.118970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa7439-2c9c-4ce0-9986-631b7971447f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9255edea8b504727a587443e124f3e7a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:52.121678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa7439-2c9c-4ce0-9986-631b7971447f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:52.259583 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cbfa7439-2c9c-4ce0-9986-631b7971447f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9255edea8b504727a587443e124f3e7a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:52.304676 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cbfa7439-2c9c-4ce0-9986-631b7971447f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9255edea8b504727a587443e124f3e7a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:52.308320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cbfa7439-2c9c-4ce0-9986-631b7971447f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9255edea8b504727a587443e124f3e7a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:52.322436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cbfa7439-2c9c-4ce0-9986-631b7971447f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9255edea8b504727a587443e124f3e7a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:52.344481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cbfa7439-2c9c-4ce0-9986-631b7971447f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9255edea8b504727a587443e124f3e7a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:52.379411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1129/2264] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:39:51 2022] DELETE /identity/v3/users/9255edea8b504727a587443e124f3e7a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:53.139416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea990ca-a2d8-43da-9460-8b082ae651bb service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:53.166651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea990ca-a2d8-43da-9460-8b082ae651bb service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:53.348105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd589fe-484c-4ce7-b76b-7ca188de9198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:53.348958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd589fe-484c-4ce7-b76b-7ca188de9198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:53.349475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd589fe-484c-4ce7-b76b-7ca188de9198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:53.349712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd589fe-484c-4ce7-b76b-7ca188de9198 admin admin] PATH_INFO: `/v3/projects/a5b0cb87dfc4461e9911c7e62b6ab959` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:53.351418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd589fe-484c-4ce7-b76b-7ca188de9198 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b0cb87dfc4461e9911c7e62b6ab959)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:53.353766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd589fe-484c-4ce7-b76b-7ca188de9198 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:53.452261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3dd589fe-484c-4ce7-b76b-7ca188de9198 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b0cb87dfc4461e9911c7e62b6ab959', 'request_id': 'req-3dd589fe-484c-4ce7-b76b-7ca188de9198'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:53.476387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3dd589fe-484c-4ce7-b76b-7ca188de9198 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b0cb87dfc4461e9911c7e62b6ab959', 'request_id': 'req-3dd589fe-484c-4ce7-b76b-7ca188de9198'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:53.488441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3dd589fe-484c-4ce7-b76b-7ca188de9198 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b0cb87dfc4461e9911c7e62b6ab959', 'request_id': 'req-3dd589fe-484c-4ce7-b76b-7ca188de9198'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:53.517977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3dd589fe-484c-4ce7-b76b-7ca188de9198 admin admin] The token cache is being invalidate because project a5b0cb87dfc4461e9911c7e62b6ab959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:39:53.518492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3dd589fe-484c-4ce7-b76b-7ca188de9198 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:53.520082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3dd589fe-484c-4ce7-b76b-7ca188de9198 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:39:53.522235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1136/2265] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:39:53 2022] DELETE /identity/v3/projects/a5b0cb87dfc4461e9911c7e62b6ab959 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:53.526787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c718f99-8754-4beb-b8d5-1bc8ab29ee27 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:53.533161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c718f99-8754-4beb-b8d5-1bc8ab29ee27 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:53.533461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c718f99-8754-4beb-b8d5-1bc8ab29ee27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:53.587556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5c718f99-8754-4beb-b8d5-1bc8ab29ee27 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:53.589048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5c718f99-8754-4beb-b8d5-1bc8ab29ee27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:53.702031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:53.779431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1130/2266] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:53.789269 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af036a6-ec29-46d8-96b7-e3605d13670e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:53.789648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af036a6-ec29-46d8-96b7-e3605d13670e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:53.790041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af036a6-ec29-46d8-96b7-e3605d13670e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:53.836664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1af036a6-ec29-46d8-96b7-e3605d13670e None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:53.841327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1af036a6-ec29-46d8-96b7-e3605d13670e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:53.853013 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1137/2267] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:53.856722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c718f99-8754-4beb-b8d5-1bc8ab29ee27 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:53.875932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c718f99-8754-4beb-b8d5-1bc8ab29ee27 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:53.888203 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c6a1c-a581-47d8-968c-58be26654ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:53.892849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c6a1c-a581-47d8-968c-58be26654ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:53.893381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c6a1c-a581-47d8-968c-58be26654ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:53.893747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c6a1c-a581-47d8-968c-58be26654ae9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:53.894557 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c6a1c-a581-47d8-968c-58be26654ae9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:39:53.896288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c6a1c-a581-47d8-968c-58be26654ae9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:53.898107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c6a1c-a581-47d8-968c-58be26654ae9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:53.947306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1131/2268] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:39:53.952214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af036a6-ec29-46d8-96b7-e3605d13670e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:53.973682 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af036a6-ec29-46d8-96b7-e3605d13670e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:53.991867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b82ed-46ae-4a85-b138-5908e0abfa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:53.992526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b82ed-46ae-4a85-b138-5908e0abfa47 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:53.992817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b82ed-46ae-4a85-b138-5908e0abfa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:53.993217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b82ed-46ae-4a85-b138-5908e0abfa47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:53.994470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b82ed-46ae-4a85-b138-5908e0abfa47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:54.004059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b82ed-46ae-4a85-b138-5908e0abfa47 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:54.089690 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1138/2269] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:39:53 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:54.091365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c6a1c-a581-47d8-968c-58be26654ae9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:54.115089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c6a1c-a581-47d8-968c-58be26654ae9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:54.263357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d2344-ac1f-4063-805c-e027c49e1d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:54.264508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d2344-ac1f-4063-805c-e027c49e1d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:54.265040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d2344-ac1f-4063-805c-e027c49e1d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:54.265546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d2344-ac1f-4063-805c-e027c49e1d59 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:54.272471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d2344-ac1f-4063-805c-e027c49e1d59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:54.274744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d2344-ac1f-4063-805c-e027c49e1d59 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:54.331227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1132/2270] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:39:54 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:39:54.332508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b82ed-46ae-4a85-b138-5908e0abfa47 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:54.349282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b82ed-46ae-4a85-b138-5908e0abfa47 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:54.364536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954e029-45eb-472f-8c6d-aa03193ad355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:54.365555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954e029-45eb-472f-8c6d-aa03193ad355 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:54.365848 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954e029-45eb-472f-8c6d-aa03193ad355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:54.366133 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954e029-45eb-472f-8c6d-aa03193ad355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:54.367244 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954e029-45eb-472f-8c6d-aa03193ad355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:54.372114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954e029-45eb-472f-8c6d-aa03193ad355 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:54.408584 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d2344-ac1f-4063-805c-e027c49e1d59 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:54.420223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1139/2271] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:54.427919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d2344-ac1f-4063-805c-e027c49e1d59 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:54.436304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc17829-5f8d-466f-8e4d-d29bbf9174c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:54.437290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc17829-5f8d-466f-8e4d-d29bbf9174c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:54.438467 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc17829-5f8d-466f-8e4d-d29bbf9174c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:54.439058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc17829-5f8d-466f-8e4d-d29bbf9174c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:54.440348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc17829-5f8d-466f-8e4d-d29bbf9174c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:54.445907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc17829-5f8d-466f-8e4d-d29bbf9174c2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:54.466329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0f87a-1fd5-49e6-b487-222067fa1504 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:54.466626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0f87a-1fd5-49e6-b487-222067fa1504 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:54.466918 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0f87a-1fd5-49e6-b487-222067fa1504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:54.486662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1133/2272] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:39:54.487675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc17829-5f8d-466f-8e4d-d29bbf9174c2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:54.514133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc17829-5f8d-466f-8e4d-d29bbf9174c2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:54.523855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37574029-1a75-4772-a8e1-ea2e898738a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:54.525498 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37574029-1a75-4772-a8e1-ea2e898738a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:54.527031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37574029-1a75-4772-a8e1-ea2e898738a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:54.527589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37574029-1a75-4772-a8e1-ea2e898738a9 admin admin] PATH_INFO: `/v3/projects/0c4f376b51ab4ee59a24d322de87e131/users/d89971a7f5f64304b675a0fc2ae2b002/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:54.532209 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7cb0f87a-1fd5-49e6-b487-222067fa1504 None None] MFA Rules not processed for user `5d7cf9c678b44bc9876e2c1efaa7277d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:54.534469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7cb0f87a-1fd5-49e6-b487-222067fa1504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:54.581326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:54.585723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37574029-1a75-4772-a8e1-ea2e898738a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4f376b51ab4ee59a24d322de87e131, user_id=d89971a7f5f64304b675a0fc2ae2b002, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:54.592302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37574029-1a75-4772-a8e1-ea2e898738a9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:54.644019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1134/2273] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:39:54 2022] PUT /identity/v3/projects/0c4f376b51ab4ee59a24d322de87e131/users/d89971a7f5f64304b675a0fc2ae2b002/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:39:54.661798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5952a17-b862-43b9-905e-64a6b02f2f4e None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:54.662098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5952a17-b862-43b9-905e-64a6b02f2f4e None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:54.662388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5952a17-b862-43b9-905e-64a6b02f2f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:54.733149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1140/2274] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:54.739315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a5952a17-b862-43b9-905e-64a6b02f2f4e None None] MFA Rules not processed for user `d89971a7f5f64304b675a0fc2ae2b002`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:39:54.740575 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a5952a17-b862-43b9-905e-64a6b02f2f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:54.742753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0f87a-1fd5-49e6-b487-222067fa1504 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:54.744573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7cb0f87a-1fd5-49e6-b487-222067fa1504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:54.904725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1135/2275] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:39:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:39:54.931024 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0f87a-1fd5-49e6-b487-222067fa1504 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:54.932206 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5952a17-b862-43b9-905e-64a6b02f2f4e None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:39:54.939480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a5952a17-b862-43b9-905e-64a6b02f2f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:55.014918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:55.123083 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5952a17-b862-43b9-905e-64a6b02f2f4e None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:39:55.124700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46883d-c995-4591-a7c2-0162e3926261 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:55.125409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46883d-c995-4591-a7c2-0162e3926261 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:55.126329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46883d-c995-4591-a7c2-0162e3926261 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:55.126329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46883d-c995-4591-a7c2-0162e3926261 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:55.132357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4c46883d-c995-4591-a7c2-0162e3926261 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:39:55.133536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8267b77-25df-44d2-ae39-b7dff8e95051 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f57fa108c3d478a9976f4aea5f33bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:39:55.135546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8267b77-25df-44d2-ae39-b7dff8e95051 service glance] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:39:55.136462 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8267b77-25df-44d2-ae39-b7dff8e95051 service glance] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:39:55.136680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8267b77-25df-44d2-ae39-b7dff8e95051 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:39:55.158129 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8267b77-25df-44d2-ae39-b7dff8e95051 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:55.159667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8267b77-25df-44d2-ae39-b7dff8e95051 service glance] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:55.192569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1136/2276] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:39:55.272800 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:39:55.313805 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46883d-c995-4591-a7c2-0162e3926261 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:39:55.317571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46883d-c995-4591-a7c2-0162e3926261 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:39:55.348460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1141/2277] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:39:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:40:00.735108 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a101d-49b2-4db6-8616-f7908a8125ec None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:00.735917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a101d-49b2-4db6-8616-f7908a8125ec None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:00.735917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a101d-49b2-4db6-8616-f7908a8125ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:00.762366 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ba6a101d-49b2-4db6-8616-f7908a8125ec None None] MFA Rules not processed for user `db6ac4cf9dd84e338f944b4905dcdb50`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:00.763810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ba6a101d-49b2-4db6-8616-f7908a8125ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:00.854633 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1137/2278] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:40:05.341368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=8 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:06.518805 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46883d-c995-4591-a7c2-0162e3926261 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:06.521073 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4c46883d-c995-4591-a7c2-0162e3926261 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:06.600937 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46883d-c995-4591-a7c2-0162e3926261 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:06.606646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:06.607328 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:06.607618 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:06.607902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] PATH_INFO: `/v3/users/ff7e97e785a54bf7845179fae2a6badb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:06.609089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff7e97e785a54bf7845179fae2a6badb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:06.611399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:06.669178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff7e97e785a54bf7845179fae2a6badb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:06.677319 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff7e97e785a54bf7845179fae2a6badb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:06.690745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff7e97e785a54bf7845179fae2a6badb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:06.691442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff7e97e785a54bf7845179fae2a6badb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:06.692093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff7e97e785a54bf7845179fae2a6badb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:06.706462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1142/2279] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:40:06 2022] DELETE /identity/v3/users/ff7e97e785a54bf7845179fae2a6badb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:40:06.711374 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d25c5-5a3d-4e2b-9b76-2318a233dad8 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:06.712534 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d25c5-5a3d-4e2b-9b76-2318a233dad8 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:06.712763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d25c5-5a3d-4e2b-9b76-2318a233dad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:06.737163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-764d25c5-5a3d-4e2b-9b76-2318a233dad8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:06.741010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-764d25c5-5a3d-4e2b-9b76-2318a233dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:06.809874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1138/2280] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:40:06.819892 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:06.821362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:06.912772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ddd94-a340-4027-b0f3-3db63183d571 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:06.918693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f348a-78f2-41fa-b840-e4ba21ce8790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:06.919372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f348a-78f2-41fa-b840-e4ba21ce8790 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:06.920544 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f348a-78f2-41fa-b840-e4ba21ce8790 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:06.920544 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f348a-78f2-41fa-b840-e4ba21ce8790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:06.940350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f348a-78f2-41fa-b840-e4ba21ce8790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:06.942355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f348a-78f2-41fa-b840-e4ba21ce8790 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:06.957323 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1143/2281] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:40:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:40:07.264063 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d5c6e-fb70-481b-865a-f95e3e0b9210 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:07.264634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d5c6e-fb70-481b-865a-f95e3e0b9210 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:07.265168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d5c6e-fb70-481b-865a-f95e3e0b9210 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:07.267298 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1139/2282] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:40:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:40:07.273740 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a41a8-9f65-42f0-bb33-4a0125304518 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:07.274124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a41a8-9f65-42f0-bb33-4a0125304518 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:07.274480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a41a8-9f65-42f0-bb33-4a0125304518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:07.302434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-008a41a8-9f65-42f0-bb33-4a0125304518 None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:07.303730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-008a41a8-9f65-42f0-bb33-4a0125304518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:07.328071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:07.410109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d5c6e-fb70-481b-865a-f95e3e0b9210 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:07.412461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5d9d5c6e-fb70-481b-865a-f95e3e0b9210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:07.419311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1144/2283] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:40:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:07.443383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d5c6e-fb70-481b-865a-f95e3e0b9210 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:07.448670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba80463-c692-4f74-a08f-4362cc55140e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:07.449310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80463-c692-4f74-a08f-4362cc55140e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:07.449780 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80463-c692-4f74-a08f-4362cc55140e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:07.450056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80463-c692-4f74-a08f-4362cc55140e admin admin] PATH_INFO: `/v3/projects/5ffec2e24e134e6188ab7103be544972` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:07.451258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba80463-c692-4f74-a08f-4362cc55140e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ffec2e24e134e6188ab7103be544972)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:07.453166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba80463-c692-4f74-a08f-4362cc55140e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:07.521705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bba80463-c692-4f74-a08f-4362cc55140e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ffec2e24e134e6188ab7103be544972', 'request_id': 'req-bba80463-c692-4f74-a08f-4362cc55140e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:07.527790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bba80463-c692-4f74-a08f-4362cc55140e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ffec2e24e134e6188ab7103be544972', 'request_id': 'req-bba80463-c692-4f74-a08f-4362cc55140e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:07.530114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:07.535295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bba80463-c692-4f74-a08f-4362cc55140e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ffec2e24e134e6188ab7103be544972', 'request_id': 'req-bba80463-c692-4f74-a08f-4362cc55140e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:07.549718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bba80463-c692-4f74-a08f-4362cc55140e admin admin] The token cache is being invalidate because project 5ffec2e24e134e6188ab7103be544972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:40:07.550016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bba80463-c692-4f74-a08f-4362cc55140e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:07.550704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-bba80463-c692-4f74-a08f-4362cc55140e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:07.552541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1140/2284] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:40:07 2022] DELETE /identity/v3/projects/5ffec2e24e134e6188ab7103be544972 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:40:07.557694 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf977b8-d0b1-4db9-8c8b-cd0874147ccd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:07.558183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf977b8-d0b1-4db9-8c8b-cd0874147ccd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:07.558468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf977b8-d0b1-4db9-8c8b-cd0874147ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:07.590546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-acf977b8-d0b1-4db9-8c8b-cd0874147ccd None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:07.591745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-acf977b8-d0b1-4db9-8c8b-cd0874147ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:07.677466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1145/2285] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:07.685167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbe58e-2c30-4028-ae5f-a1432566bd0a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:07.685444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbe58e-2c30-4028-ae5f-a1432566bd0a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:07.685720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbe58e-2c30-4028-ae5f-a1432566bd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:07.711018 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-05cbe58e-2c30-4028-ae5f-a1432566bd0a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:07.712620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-05cbe58e-2c30-4028-ae5f-a1432566bd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:07.722281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1141/2286] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:07.726400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf977b8-d0b1-4db9-8c8b-cd0874147ccd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:07.738980 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf977b8-d0b1-4db9-8c8b-cd0874147ccd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:07.743922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffae024-c4aa-4e2b-b31f-b111dcb8a0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:07.744617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffae024-c4aa-4e2b-b31f-b111dcb8a0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:07.745442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffae024-c4aa-4e2b-b31f-b111dcb8a0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:07.745442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffae024-c4aa-4e2b-b31f-b111dcb8a0c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:07.746078 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffae024-c4aa-4e2b-b31f-b111dcb8a0c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:40:07.746455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffae024-c4aa-4e2b-b31f-b111dcb8a0c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:07.748168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffae024-c4aa-4e2b-b31f-b111dcb8a0c4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:07.761969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1146/2287] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:40:07.766239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbe58e-2c30-4028-ae5f-a1432566bd0a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:07.775610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbe58e-2c30-4028-ae5f-a1432566bd0a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:07.780747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ef207-b654-45ad-a214-b49262331667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:07.781387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ef207-b654-45ad-a214-b49262331667 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:07.781846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ef207-b654-45ad-a214-b49262331667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:07.782151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ef207-b654-45ad-a214-b49262331667 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:07.783532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ef207-b654-45ad-a214-b49262331667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:07.785799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ef207-b654-45ad-a214-b49262331667 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:07.821216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1142/2288] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:40:07 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:40:07.822659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffae024-c4aa-4e2b-b31f-b111dcb8a0c4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:07.832984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffae024-c4aa-4e2b-b31f-b111dcb8a0c4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:07.895648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa077c28-f3e7-4b11-bc51-0335bf2caf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:07.896410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa077c28-f3e7-4b11-bc51-0335bf2caf26 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:07.896700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa077c28-f3e7-4b11-bc51-0335bf2caf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:07.897703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa077c28-f3e7-4b11-bc51-0335bf2caf26 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:07.897703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa077c28-f3e7-4b11-bc51-0335bf2caf26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:07.899638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa077c28-f3e7-4b11-bc51-0335bf2caf26 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:07.931178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1147/2289] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:40:07 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:40:07.934932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ef207-b654-45ad-a214-b49262331667 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:07.948248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ef207-b654-45ad-a214-b49262331667 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:07.953869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4d586-c14a-4247-81ed-28f19cdacad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:07.955155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4d586-c14a-4247-81ed-28f19cdacad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:07.955509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4d586-c14a-4247-81ed-28f19cdacad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:07.955818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4d586-c14a-4247-81ed-28f19cdacad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:07.956548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4d586-c14a-4247-81ed-28f19cdacad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:07.958278 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4d586-c14a-4247-81ed-28f19cdacad2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:07.978323 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1143/2290] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:40:07.979626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa077c28-f3e7-4b11-bc51-0335bf2caf26 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:07.991759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa077c28-f3e7-4b11-bc51-0335bf2caf26 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:07.997195 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394943b9-af0b-46bf-8b50-753ea8b315c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:07.997895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394943b9-af0b-46bf-8b50-753ea8b315c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:07.998242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394943b9-af0b-46bf-8b50-753ea8b315c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:07.998589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394943b9-af0b-46bf-8b50-753ea8b315c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:07.999448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394943b9-af0b-46bf-8b50-753ea8b315c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:08.001777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394943b9-af0b-46bf-8b50-753ea8b315c6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:08.015026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1148/2291] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:40:08.019594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4d586-c14a-4247-81ed-28f19cdacad2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:08.030511 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4d586-c14a-4247-81ed-28f19cdacad2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:08.037472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974bb9d-5c9c-463b-a189-4ffc307fafb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:08.037472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974bb9d-5c9c-463b-a189-4ffc307fafb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:08.037472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974bb9d-5c9c-463b-a189-4ffc307fafb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:08.050328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974bb9d-5c9c-463b-a189-4ffc307fafb1 admin admin] PATH_INFO: `/v3/projects/4a76209de52146bf8440fddcdd00b284/users/42a2bc662fa4405791532ebf2bce909c/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:08.099165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974bb9d-5c9c-463b-a189-4ffc307fafb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a76209de52146bf8440fddcdd00b284, user_id=42a2bc662fa4405791532ebf2bce909c, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:08.101877 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974bb9d-5c9c-463b-a189-4ffc307fafb1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:08.119953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1144/2292] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:40:08 2022] PUT /identity/v3/projects/4a76209de52146bf8440fddcdd00b284/users/42a2bc662fa4405791532ebf2bce909c/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:40:08.125171 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2c4e7-5850-44a6-87de-8a302401fd0d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:08.125484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2c4e7-5850-44a6-87de-8a302401fd0d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:08.125765 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2c4e7-5850-44a6-87de-8a302401fd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:08.146864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-51a2c4e7-5850-44a6-87de-8a302401fd0d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:08.148130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-51a2c4e7-5850-44a6-87de-8a302401fd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:08.161405 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:08.216641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1149/2293] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:08.227602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974bb9d-5c9c-463b-a189-4ffc307fafb1 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:08.229556 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6974bb9d-5c9c-463b-a189-4ffc307fafb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:08.289965 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974bb9d-5c9c-463b-a189-4ffc307fafb1 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:08.294552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152a62d-317d-4f1d-a681-e317726b8877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:08.295217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152a62d-317d-4f1d-a681-e317726b8877 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:08.295736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152a62d-317d-4f1d-a681-e317726b8877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:08.296030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152a62d-317d-4f1d-a681-e317726b8877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:08.307034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152a62d-317d-4f1d-a681-e317726b8877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:08.308787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152a62d-317d-4f1d-a681-e317726b8877 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:08.319550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1145/2294] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:40:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:40:09.215565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901a851-c417-4fcb-8bd9-8b847a5eada6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:09.216232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901a851-c417-4fcb-8bd9-8b847a5eada6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:09.216232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901a851-c417-4fcb-8bd9-8b847a5eada6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:09.259502 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9901a851-c417-4fcb-8bd9-8b847a5eada6 None None] MFA Rules not processed for user `42a2bc662fa4405791532ebf2bce909c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:09.260750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9901a851-c417-4fcb-8bd9-8b847a5eada6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:09.338060 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1150/2295] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:09.352017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152a62d-317d-4f1d-a681-e317726b8877 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:09.353516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a152a62d-317d-4f1d-a681-e317726b8877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:09.430867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152a62d-317d-4f1d-a681-e317726b8877 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:09.435541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757b4be-4581-45c2-9acf-a3766d5f50e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:09.436704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757b4be-4581-45c2-9acf-a3766d5f50e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:09.436841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757b4be-4581-45c2-9acf-a3766d5f50e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:09.437310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757b4be-4581-45c2-9acf-a3766d5f50e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:09.448053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757b4be-4581-45c2-9acf-a3766d5f50e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:09.449674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757b4be-4581-45c2-9acf-a3766d5f50e0 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:09.461451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1146/2296] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:40:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:40:10.234240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901a851-c417-4fcb-8bd9-8b847a5eada6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:10.236539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9901a851-c417-4fcb-8bd9-8b847a5eada6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:10.293545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901a851-c417-4fcb-8bd9-8b847a5eada6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:10.297057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75a8f1-9898-4e9d-9fc1-01b6fc130732 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f57fa108c3d478a9976f4aea5f33bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:10.298203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75a8f1-9898-4e9d-9fc1-01b6fc130732 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:10.298497 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75a8f1-9898-4e9d-9fc1-01b6fc130732 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:10.298774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75a8f1-9898-4e9d-9fc1-01b6fc130732 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:10.300557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5d75a8f1-9898-4e9d-9fc1-01b6fc130732 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:10.357080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:10.372662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75a8f1-9898-4e9d-9fc1-01b6fc130732 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:10.374118 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75a8f1-9898-4e9d-9fc1-01b6fc130732 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:10.385447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1151/2297] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:40:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:40:10.458791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044f900-6f30-4051-b4be-18b961d98a05 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:10.459368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044f900-6f30-4051-b4be-18b961d98a05 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:10.459484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044f900-6f30-4051-b4be-18b961d98a05 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:10.460783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1147/2298] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:40:10.465525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c92283-b1b5-41ac-aebd-add760e260d5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:10.465823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c92283-b1b5-41ac-aebd-add760e260d5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:10.466104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c92283-b1b5-41ac-aebd-add760e260d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:10.487455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-88c92283-b1b5-41ac-aebd-add760e260d5 None None] MFA Rules not processed for user `138ede2854894a3b995435980965911c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:10.488600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-88c92283-b1b5-41ac-aebd-add760e260d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:10.495113 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1152/2299] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:40:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:10.507327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044f900-6f30-4051-b4be-18b961d98a05 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:10.518834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044f900-6f30-4051-b4be-18b961d98a05 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:10.522470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9054ad-820b-4eb9-a3cd-6b96fcfde499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:10.523155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9054ad-820b-4eb9-a3cd-6b96fcfde499 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:10.523402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9054ad-820b-4eb9-a3cd-6b96fcfde499 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:10.523683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9054ad-820b-4eb9-a3cd-6b96fcfde499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:10.530723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9054ad-820b-4eb9-a3cd-6b96fcfde499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:10.532857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9054ad-820b-4eb9-a3cd-6b96fcfde499 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:10.540688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1148/2300] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:40:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:40:10.656620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f800363-0c6f-47a8-8449-f2fc986b241d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:10.656909 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f800363-0c6f-47a8-8449-f2fc986b241d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:10.657251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f800363-0c6f-47a8-8449-f2fc986b241d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:10.658746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1153/2301] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:40:10.663466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1b296-ca58-4490-9319-bc5a1e87807a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:10.663765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1b296-ca58-4490-9319-bc5a1e87807a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:10.664049 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1b296-ca58-4490-9319-bc5a1e87807a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:10.685400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9ef1b296-ca58-4490-9319-bc5a1e87807a None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:10.686768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9ef1b296-ca58-4490-9319-bc5a1e87807a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:10.698065 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1149/2302] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:40:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:10.709146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f800363-0c6f-47a8-8449-f2fc986b241d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:10.720707 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f800363-0c6f-47a8-8449-f2fc986b241d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:10.724340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc06f65-f9d9-458c-9f57-ee25d0372954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:10.724949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc06f65-f9d9-458c-9f57-ee25d0372954 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:10.725232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc06f65-f9d9-458c-9f57-ee25d0372954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:10.725510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc06f65-f9d9-458c-9f57-ee25d0372954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:10.737554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc06f65-f9d9-458c-9f57-ee25d0372954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:10.739043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc06f65-f9d9-458c-9f57-ee25d0372954 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:10.750122 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1154/2303] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:40:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:40:10.882766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63492279-e529-468b-a302-e2cb952d3e4c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:10.883169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63492279-e529-468b-a302-e2cb952d3e4c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:10.883516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63492279-e529-468b-a302-e2cb952d3e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:10.903672 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-63492279-e529-468b-a302-e2cb952d3e4c None None] MFA Rules not processed for user `42a2bc662fa4405791532ebf2bce909c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:10.904885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-63492279-e529-468b-a302-e2cb952d3e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:10.913364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1150/2304] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:12.165275 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc06f65-f9d9-458c-9f57-ee25d0372954 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:12.167541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1dc06f65-f9d9-458c-9f57-ee25d0372954 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:12.263991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc06f65-f9d9-458c-9f57-ee25d0372954 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:12.270449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcf41a-df86-4d37-8a8a-3df7d816f451 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928c5d1284b74f1291c031323bb03a20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:12.271115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcf41a-df86-4d37-8a8a-3df7d816f451 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:12.271415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcf41a-df86-4d37-8a8a-3df7d816f451 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:12.271689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcf41a-df86-4d37-8a8a-3df7d816f451 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:12.275254 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-45dcf41a-df86-4d37-8a8a-3df7d816f451 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:12.289214 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcf41a-df86-4d37-8a8a-3df7d816f451 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:12.291452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcf41a-df86-4d37-8a8a-3df7d816f451 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:12.303601 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1155/2305] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:40:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:40:15.387720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b892a-65f2-44d8-ae70-71569c0f1479 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:15.388182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b892a-65f2-44d8-ae70-71569c0f1479 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:15.388289 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b892a-65f2-44d8-ae70-71569c0f1479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:15.396674 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:15.409683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-956b892a-65f2-44d8-ae70-71569c0f1479 None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:15.411170 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-956b892a-65f2-44d8-ae70-71569c0f1479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:15.473665 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1151/2306] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:40:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:15.495505 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcf41a-df86-4d37-8a8a-3df7d816f451 service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:15.497105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-45dcf41a-df86-4d37-8a8a-3df7d816f451 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:15.594087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcf41a-df86-4d37-8a8a-3df7d816f451 service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:15.604192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fdeb7-7efa-47d3-a587-a15da5c5f57c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:15.605165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fdeb7-7efa-47d3-a587-a15da5c5f57c service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:15.605588 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fdeb7-7efa-47d3-a587-a15da5c5f57c service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:15.606305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fdeb7-7efa-47d3-a587-a15da5c5f57c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:15.618672 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365fdeb7-7efa-47d3-a587-a15da5c5f57c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:15.620428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365fdeb7-7efa-47d3-a587-a15da5c5f57c service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:15.629951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1156/2307] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:40:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:40:25.465921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=23 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:25.626166 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=77 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:27.955130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece992b1-ab44-40ce-8cdb-9bb4dbc3ac26 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:27.955591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece992b1-ab44-40ce-8cdb-9bb4dbc3ac26 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:27.955696 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece992b1-ab44-40ce-8cdb-9bb4dbc3ac26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:27.976054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ece992b1-ab44-40ce-8cdb-9bb4dbc3ac26 None None] MFA Rules not processed for user `890be2e19b43473caa79b2dc889f8436`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:27.977256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ece992b1-ab44-40ce-8cdb-9bb4dbc3ac26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:28.033248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1152/2308] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:28.494577 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fdeb7-7efa-47d3-a587-a15da5c5f57c service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:28.500197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-365fdeb7-7efa-47d3-a587-a15da5c5f57c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:28.515541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fdeb7-7efa-47d3-a587-a15da5c5f57c service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:28.524706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:28.526514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:28.528093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:28.528093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] PATH_INFO: `/v3/users/d89971a7f5f64304b675a0fc2ae2b002` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:28.531864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89971a7f5f64304b675a0fc2ae2b002)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:28.534991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:28.596531 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89971a7f5f64304b675a0fc2ae2b002'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:28.605073 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89971a7f5f64304b675a0fc2ae2b002'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:28.619421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89971a7f5f64304b675a0fc2ae2b002'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:28.620144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89971a7f5f64304b675a0fc2ae2b002'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:28.620887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89971a7f5f64304b675a0fc2ae2b002'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:28.634127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1157/2309] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:40:28 2022] DELETE /identity/v3/users/d89971a7f5f64304b675a0fc2ae2b002 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:40:28.638634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc428904-1dc8-4343-87cc-ad1825b91268 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:28.638928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc428904-1dc8-4343-87cc-ad1825b91268 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:28.639245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc428904-1dc8-4343-87cc-ad1825b91268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:28.660188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-dc428904-1dc8-4343-87cc-ad1825b91268 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:28.661552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dc428904-1dc8-4343-87cc-ad1825b91268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:28.734930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1153/2310] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:28.745709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:28.747193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:28.810587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042744b-5e25-42cc-840a-2b764ddd1905 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:28.814406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be849c97-b5e7-48c7-93d7-8b4a1311c2a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:28.815031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be849c97-b5e7-48c7-93d7-8b4a1311c2a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:28.815348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be849c97-b5e7-48c7-93d7-8b4a1311c2a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:28.815628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be849c97-b5e7-48c7-93d7-8b4a1311c2a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:28.825137 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be849c97-b5e7-48c7-93d7-8b4a1311c2a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:28.826951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be849c97-b5e7-48c7-93d7-8b4a1311c2a9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:28.840044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1158/2311] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:40:29.340083 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc428904-1dc8-4343-87cc-ad1825b91268 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:29.341498 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-dc428904-1dc8-4343-87cc-ad1825b91268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:29.357546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc428904-1dc8-4343-87cc-ad1825b91268 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:29.368565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798e145-a72f-41ef-8148-8b7b5af4f78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:29.369341 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798e145-a72f-41ef-8148-8b7b5af4f78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:29.370640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798e145-a72f-41ef-8148-8b7b5af4f78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:29.371177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798e145-a72f-41ef-8148-8b7b5af4f78c admin admin] PATH_INFO: `/v3/projects/0c4f376b51ab4ee59a24d322de87e131` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:29.373254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798e145-a72f-41ef-8148-8b7b5af4f78c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4f376b51ab4ee59a24d322de87e131)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:29.377516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798e145-a72f-41ef-8148-8b7b5af4f78c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:29.437259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0798e145-a72f-41ef-8148-8b7b5af4f78c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4f376b51ab4ee59a24d322de87e131', 'request_id': 'req-0798e145-a72f-41ef-8148-8b7b5af4f78c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:29.445524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0798e145-a72f-41ef-8148-8b7b5af4f78c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4f376b51ab4ee59a24d322de87e131', 'request_id': 'req-0798e145-a72f-41ef-8148-8b7b5af4f78c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:29.452006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0798e145-a72f-41ef-8148-8b7b5af4f78c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4f376b51ab4ee59a24d322de87e131', 'request_id': 'req-0798e145-a72f-41ef-8148-8b7b5af4f78c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:29.474883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0798e145-a72f-41ef-8148-8b7b5af4f78c admin admin] The token cache is being invalidate because project 0c4f376b51ab4ee59a24d322de87e131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:40:29.475707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0798e145-a72f-41ef-8148-8b7b5af4f78c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:29.476531 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0798e145-a72f-41ef-8148-8b7b5af4f78c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:29.482388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1154/2312] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:40:29 2022] DELETE /identity/v3/projects/0c4f376b51ab4ee59a24d322de87e131 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:40:29.483992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eda050-4314-4670-9dcf-e46361f92afd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:29.484290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eda050-4314-4670-9dcf-e46361f92afd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:29.484570 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eda050-4314-4670-9dcf-e46361f92afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:29.507013 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c0eda050-4314-4670-9dcf-e46361f92afd None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:29.508290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0eda050-4314-4670-9dcf-e46361f92afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:29.592564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1159/2313] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:40:29.602390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f356c61-b12d-4bf7-a6be-4967c9b1acf7 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:29.602718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f356c61-b12d-4bf7-a6be-4967c9b1acf7 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:29.603267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f356c61-b12d-4bf7-a6be-4967c9b1acf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:29.629830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1f356c61-b12d-4bf7-a6be-4967c9b1acf7 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:29.632229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1f356c61-b12d-4bf7-a6be-4967c9b1acf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:29.646229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1155/2314] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:29.647513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eda050-4314-4670-9dcf-e46361f92afd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:29.661204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eda050-4314-4670-9dcf-e46361f92afd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:29.670387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d16b0-7220-4337-9303-4b65168efc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:29.671231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d16b0-7220-4337-9303-4b65168efc29 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:29.671788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d16b0-7220-4337-9303-4b65168efc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:29.672139 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d16b0-7220-4337-9303-4b65168efc29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:29.672958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d16b0-7220-4337-9303-4b65168efc29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:40:29.673653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d16b0-7220-4337-9303-4b65168efc29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:29.675567 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d16b0-7220-4337-9303-4b65168efc29 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:29.699476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1160/2315] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:40:29.701757 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f356c61-b12d-4bf7-a6be-4967c9b1acf7 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:29.717245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f356c61-b12d-4bf7-a6be-4967c9b1acf7 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:29.724044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b604e1-b38c-4c22-aeb7-fa57a4848d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:29.724697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b604e1-b38c-4c22-aeb7-fa57a4848d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:29.735258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b604e1-b38c-4c22-aeb7-fa57a4848d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:29.735860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b604e1-b38c-4c22-aeb7-fa57a4848d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:29.737738 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b604e1-b38c-4c22-aeb7-fa57a4848d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:29.743672 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b604e1-b38c-4c22-aeb7-fa57a4848d0c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:29.780427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:29.790884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1156/2316] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:40:29 2022] POST /identity/v3/projects => generated 389 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:40:29.794027 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d16b0-7220-4337-9303-4b65168efc29 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:29.799980 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:29.806085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d16b0-7220-4337-9303-4b65168efc29 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:29.869494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669178b-c20d-4ddf-bfd5-73f693a25286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:29.870106 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669178b-c20d-4ddf-bfd5-73f693a25286 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:29.870390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669178b-c20d-4ddf-bfd5-73f693a25286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:29.870665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669178b-c20d-4ddf-bfd5-73f693a25286 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:29.871382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669178b-c20d-4ddf-bfd5-73f693a25286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:29.873332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669178b-c20d-4ddf-bfd5-73f693a25286 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:29.902882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1161/2317] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:40:29 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:40:29.904154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b604e1-b38c-4c22-aeb7-fa57a4848d0c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:29.915489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b604e1-b38c-4c22-aeb7-fa57a4848d0c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:29.920810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512426bf-58b5-488d-9d00-6a355b9d1c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:29.921424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512426bf-58b5-488d-9d00-6a355b9d1c5f admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:29.921755 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512426bf-58b5-488d-9d00-6a355b9d1c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:29.922017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512426bf-58b5-488d-9d00-6a355b9d1c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:29.922714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512426bf-58b5-488d-9d00-6a355b9d1c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:29.924926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512426bf-58b5-488d-9d00-6a355b9d1c5f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:29.938349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1157/2318] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:40:29.940207 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669178b-c20d-4ddf-bfd5-73f693a25286 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:29.951042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669178b-c20d-4ddf-bfd5-73f693a25286 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:29.956429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56906212-7b2a-40ff-962b-0ea5270cc490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:29.957055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56906212-7b2a-40ff-962b-0ea5270cc490 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:29.957340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56906212-7b2a-40ff-962b-0ea5270cc490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:29.957649 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56906212-7b2a-40ff-962b-0ea5270cc490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:29.958377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56906212-7b2a-40ff-962b-0ea5270cc490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:29.962005 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56906212-7b2a-40ff-962b-0ea5270cc490 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:29.975141 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1162/2319] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:40:29.977034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512426bf-58b5-488d-9d00-6a355b9d1c5f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:29.989178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512426bf-58b5-488d-9d00-6a355b9d1c5f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:29.994441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ec9d1-63a8-4b38-a2cb-ceeabb388713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:29.994884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec9d1-63a8-4b38-a2cb-ceeabb388713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:29.995206 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec9d1-63a8-4b38-a2cb-ceeabb388713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:29.995485 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec9d1-63a8-4b38-a2cb-ceeabb388713 admin admin] PATH_INFO: `/v3/projects/5e1248ec7ee148b2ac8aa52a9e4f7049/users/0b3574177ef349a69f908668d892cbbc/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:30.013839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ec9d1-63a8-4b38-a2cb-ceeabb388713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1248ec7ee148b2ac8aa52a9e4f7049, user_id=0b3574177ef349a69f908668d892cbbc, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:30.017351 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ec9d1-63a8-4b38-a2cb-ceeabb388713 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:30.042762 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1158/2320] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:40:29 2022] PUT /identity/v3/projects/5e1248ec7ee148b2ac8aa52a9e4f7049/users/0b3574177ef349a69f908668d892cbbc/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:40:30.114945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b6dec-948b-4f81-a66a-7b4ba0046234 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:30.115273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b6dec-948b-4f81-a66a-7b4ba0046234 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:30.115549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b6dec-948b-4f81-a66a-7b4ba0046234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:30.149368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-968b6dec-948b-4f81-a66a-7b4ba0046234 None None] MFA Rules not processed for user `0b3574177ef349a69f908668d892cbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:30.149654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-968b6dec-948b-4f81-a66a-7b4ba0046234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:30.241263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1163/2321] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:30.255689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ec9d1-63a8-4b38-a2cb-ceeabb388713 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:30.257913 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ee8ec9d1-63a8-4b38-a2cb-ceeabb388713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:30.343921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ec9d1-63a8-4b38-a2cb-ceeabb388713 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:30.348391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8ce17-d31e-46dd-a657-6ca69d22c90c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:30.349289 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8ce17-d31e-46dd-a657-6ca69d22c90c service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:30.349750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8ce17-d31e-46dd-a657-6ca69d22c90c service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:30.350188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8ce17-d31e-46dd-a657-6ca69d22c90c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:30.360158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8ce17-d31e-46dd-a657-6ca69d22c90c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:30.362598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8ce17-d31e-46dd-a657-6ca69d22c90c service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:30.373178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1159/2322] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:40:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:40:40.236759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:40.368875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:46.228360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cd417-bafb-4134-83d8-dae1666112b1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:46.228875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cd417-bafb-4134-83d8-dae1666112b1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:46.228970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cd417-bafb-4134-83d8-dae1666112b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:46.248867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-801cd417-bafb-4134-83d8-dae1666112b1 None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:46.250068 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-801cd417-bafb-4134-83d8-dae1666112b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:46.307643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1164/2323] 10.222.0.40 () {58 vars in 967 bytes} [Mon Aug 1 07:40:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:46.329548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8ce17-d31e-46dd-a657-6ca69d22c90c service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:46.330981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7ea8ce17-d31e-46dd-a657-6ca69d22c90c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:46.405943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8ce17-d31e-46dd-a657-6ca69d22c90c service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:46.411942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aac249-523e-4b48-a8e7-96a070797d4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:46.412558 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aac249-523e-4b48-a8e7-96a070797d4f service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:46.412838 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aac249-523e-4b48-a8e7-96a070797d4f service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:46.413112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aac249-523e-4b48-a8e7-96a070797d4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:46.422033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aac249-523e-4b48-a8e7-96a070797d4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:46.423830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aac249-523e-4b48-a8e7-96a070797d4f service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:46.432249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1160/2324] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:40:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:40:50.041152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cd417-bafb-4134-83d8-dae1666112b1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:50.110330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cd417-bafb-4134-83d8-dae1666112b1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:50.114904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:50.115586 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:50.115875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:50.116162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] PATH_INFO: `/v3/users/0b3574177ef349a69f908668d892cbbc` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:50.117341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b3574177ef349a69f908668d892cbbc)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:50.119316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:50.165043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b3574177ef349a69f908668d892cbbc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:50.171462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b3574177ef349a69f908668d892cbbc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:50.183185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b3574177ef349a69f908668d892cbbc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:50.183826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b3574177ef349a69f908668d892cbbc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:50.184463 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b3574177ef349a69f908668d892cbbc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:50.197470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1165/2325] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:40:50 2022] DELETE /identity/v3/users/0b3574177ef349a69f908668d892cbbc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:40:50.201881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f5ce1-4cec-4adc-ab74-c86b4e8cfb82 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:50.202422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f5ce1-4cec-4adc-ab74-c86b4e8cfb82 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:50.202422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f5ce1-4cec-4adc-ab74-c86b4e8cfb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:50.221516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ed4f5ce1-4cec-4adc-ab74-c86b4e8cfb82 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:50.222898 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ed4f5ce1-4cec-4adc-ab74-c86b4e8cfb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:50.283553 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1161/2326] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:50.293660 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:50.295090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:50.322452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:50.354588 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d2df7-6775-4a94-af43-eca7107f5376 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:50.358235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c667ed-b8b3-4311-9e16-e8d230e1fae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:50.358843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c667ed-b8b3-4311-9e16-e8d230e1fae1 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:50.359160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c667ed-b8b3-4311-9e16-e8d230e1fae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:50.359440 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c667ed-b8b3-4311-9e16-e8d230e1fae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:50.368480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c667ed-b8b3-4311-9e16-e8d230e1fae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:50.369849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c667ed-b8b3-4311-9e16-e8d230e1fae1 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:50.379940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1166/2327] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:40:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:40:50.663277 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f5ce1-4cec-4adc-ab74-c86b4e8cfb82 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:50.664733 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ed4f5ce1-4cec-4adc-ab74-c86b4e8cfb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:50.677495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f5ce1-4cec-4adc-ab74-c86b4e8cfb82 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:50.682546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:50.683231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:50.683520 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:50.683810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c admin admin] PATH_INFO: `/v3/projects/5e1248ec7ee148b2ac8aa52a9e4f7049` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:50.684970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1248ec7ee148b2ac8aa52a9e4f7049)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:50.686890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:50.738168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1248ec7ee148b2ac8aa52a9e4f7049', 'request_id': 'req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:50.743722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1248ec7ee148b2ac8aa52a9e4f7049', 'request_id': 'req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:50.749848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1248ec7ee148b2ac8aa52a9e4f7049', 'request_id': 'req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:50.762496 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c admin admin] The token cache is being invalidate because project 5e1248ec7ee148b2ac8aa52a9e4f7049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:40:50.762785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:50.763425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5394e9b4-8d8a-46fd-beb6-7bf8fec0321c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:40:50.764937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1162/2328] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:40:50 2022] DELETE /identity/v3/projects/5e1248ec7ee148b2ac8aa52a9e4f7049 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:40:50.770480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33561a4-5726-40f5-81ed-68e6934830ef None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:50.770766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33561a4-5726-40f5-81ed-68e6934830ef None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:50.771042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33561a4-5726-40f5-81ed-68e6934830ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:50.800152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b33561a4-5726-40f5-81ed-68e6934830ef None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:50.802829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b33561a4-5726-40f5-81ed-68e6934830ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:50.864690 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1167/2329] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:40:50.872370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce626f50-f23f-4223-bf09-6a48f815a782 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:50.872655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce626f50-f23f-4223-bf09-6a48f815a782 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:50.872931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce626f50-f23f-4223-bf09-6a48f815a782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:50.893759 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ce626f50-f23f-4223-bf09-6a48f815a782 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:50.894881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ce626f50-f23f-4223-bf09-6a48f815a782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:50.903092 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1163/2330] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:50.908147 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33561a4-5726-40f5-81ed-68e6934830ef None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:50.918467 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33561a4-5726-40f5-81ed-68e6934830ef None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:50.923118 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ae676-cd4a-4bc5-9609-90fb37f0f954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:50.923731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ae676-cd4a-4bc5-9609-90fb37f0f954 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:50.924017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ae676-cd4a-4bc5-9609-90fb37f0f954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:50.924297 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ae676-cd4a-4bc5-9609-90fb37f0f954 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:50.925167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ae676-cd4a-4bc5-9609-90fb37f0f954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:40:50.925622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ae676-cd4a-4bc5-9609-90fb37f0f954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:50.927235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ae676-cd4a-4bc5-9609-90fb37f0f954 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:50.943005 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1168/2331] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:40:50.948747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce626f50-f23f-4223-bf09-6a48f815a782 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:50.959932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce626f50-f23f-4223-bf09-6a48f815a782 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:50.966594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db460ae7-0d96-489e-a70b-7e54cd028a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:50.967389 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db460ae7-0d96-489e-a70b-7e54cd028a3e admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:50.967701 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db460ae7-0d96-489e-a70b-7e54cd028a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:50.967986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db460ae7-0d96-489e-a70b-7e54cd028a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:50.968662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db460ae7-0d96-489e-a70b-7e54cd028a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:50.970641 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db460ae7-0d96-489e-a70b-7e54cd028a3e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:50.975748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:51.000813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1164/2332] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:40:50 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:40:51.006282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ae676-cd4a-4bc5-9609-90fb37f0f954 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:51.016420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ae676-cd4a-4bc5-9609-90fb37f0f954 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:51.072847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c3e1b-9d57-4214-8862-e60985c1b19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:51.073507 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c3e1b-9d57-4214-8862-e60985c1b19d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.073737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c3e1b-9d57-4214-8862-e60985c1b19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.074020 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c3e1b-9d57-4214-8862-e60985c1b19d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.074692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c3e1b-9d57-4214-8862-e60985c1b19d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:51.076658 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c3e1b-9d57-4214-8862-e60985c1b19d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:51.101827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1169/2333] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:40:51 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:40:51.103009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db460ae7-0d96-489e-a70b-7e54cd028a3e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:51.114865 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db460ae7-0d96-489e-a70b-7e54cd028a3e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:51.119689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f83772-7680-496b-8725-9d6bdd6c7440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:51.120316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f83772-7680-496b-8725-9d6bdd6c7440 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.120740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f83772-7680-496b-8725-9d6bdd6c7440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.121039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f83772-7680-496b-8725-9d6bdd6c7440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.121771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f83772-7680-496b-8725-9d6bdd6c7440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:51.123416 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f83772-7680-496b-8725-9d6bdd6c7440 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:51.138185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1165/2334] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:40:51.139473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c3e1b-9d57-4214-8862-e60985c1b19d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:51.150831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c3e1b-9d57-4214-8862-e60985c1b19d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:51.155644 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483332c-b95a-4f43-b186-1c8b226f0702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:51.156335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483332c-b95a-4f43-b186-1c8b226f0702 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.156558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483332c-b95a-4f43-b186-1c8b226f0702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.156840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483332c-b95a-4f43-b186-1c8b226f0702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.157537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483332c-b95a-4f43-b186-1c8b226f0702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:51.159781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483332c-b95a-4f43-b186-1c8b226f0702 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:51.170901 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1170/2335] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:40:51.175340 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f83772-7680-496b-8725-9d6bdd6c7440 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:51.186109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f83772-7680-496b-8725-9d6bdd6c7440 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:51.191602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eef018-8335-4f1a-a97c-5eaa3f9aa0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:51.192279 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eef018-8335-4f1a-a97c-5eaa3f9aa0a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.192562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eef018-8335-4f1a-a97c-5eaa3f9aa0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.192841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eef018-8335-4f1a-a97c-5eaa3f9aa0a8 admin admin] PATH_INFO: `/v3/projects/a27abd6ed13c4174a4c14226fb48e146/users/6e461e65817648b0a878d8280c84dc6e/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.209376 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eef018-8335-4f1a-a97c-5eaa3f9aa0a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a27abd6ed13c4174a4c14226fb48e146, user_id=6e461e65817648b0a878d8280c84dc6e, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:51.212172 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eef018-8335-4f1a-a97c-5eaa3f9aa0a8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:51.225957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1166/2336] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:40:51 2022] PUT /identity/v3/projects/a27abd6ed13c4174a4c14226fb48e146/users/6e461e65817648b0a878d8280c84dc6e/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:40:51.230958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a674c-e294-4c4d-97ae-aad7a777a6d1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.231281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a674c-e294-4c4d-97ae-aad7a777a6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.231566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a674c-e294-4c4d-97ae-aad7a777a6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.254962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a79a674c-e294-4c4d-97ae-aad7a777a6d1 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:51.256281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a79a674c-e294-4c4d-97ae-aad7a777a6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:51.286866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:51.318185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1171/2337] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:51.327915 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eef018-8335-4f1a-a97c-5eaa3f9aa0a8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:51.329598 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c4eef018-8335-4f1a-a97c-5eaa3f9aa0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:51.386831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eef018-8335-4f1a-a97c-5eaa3f9aa0a8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:51.390463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237c07d-263d-4798-b549-e8edb3fd41a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:51.391090 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237c07d-263d-4798-b549-e8edb3fd41a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:51.391424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237c07d-263d-4798-b549-e8edb3fd41a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:51.391665 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237c07d-263d-4798-b549-e8edb3fd41a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:51.404179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237c07d-263d-4798-b549-e8edb3fd41a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:51.405661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237c07d-263d-4798-b549-e8edb3fd41a5 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:51.415940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1167/2338] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:40:55.852919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556a8a4-e174-4e8d-a492-51e4f5f3646e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:55.853522 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556a8a4-e174-4e8d-a492-51e4f5f3646e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:55.853522 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556a8a4-e174-4e8d-a492-51e4f5f3646e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:55.885638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7556a8a4-e174-4e8d-a492-51e4f5f3646e None None] MFA Rules not processed for user `6e461e65817648b0a878d8280c84dc6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:40:55.886788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7556a8a4-e174-4e8d-a492-51e4f5f3646e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:55.964605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1172/2339] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:40:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:40:55.966700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237c07d-263d-4798-b549-e8edb3fd41a5 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:55.979304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237c07d-263d-4798-b549-e8edb3fd41a5 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:55.985120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eda6b7-2343-4a60-a8d4-a83b91d6ab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:55.985886 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eda6b7-2343-4a60-a8d4-a83b91d6ab83 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:55.986166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eda6b7-2343-4a60-a8d4-a83b91d6ab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:55.986472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eda6b7-2343-4a60-a8d4-a83b91d6ab83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:55.987681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eda6b7-2343-4a60-a8d4-a83b91d6ab83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:55.989770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eda6b7-2343-4a60-a8d4-a83b91d6ab83 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.019160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1168/2340] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:40:55 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:40:56.022969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556a8a4-e174-4e8d-a492-51e4f5f3646e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:56.035834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556a8a4-e174-4e8d-a492-51e4f5f3646e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:56.116111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d73cad-2fe9-460c-9f07-2266efd2ac54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:56.116111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d73cad-2fe9-460c-9f07-2266efd2ac54 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.116111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d73cad-2fe9-460c-9f07-2266efd2ac54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.116111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d73cad-2fe9-460c-9f07-2266efd2ac54 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.116608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d73cad-2fe9-460c-9f07-2266efd2ac54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:56.116608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d73cad-2fe9-460c-9f07-2266efd2ac54 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.142202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1173/2341] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:40:56 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:40:56.146358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eda6b7-2343-4a60-a8d4-a83b91d6ab83 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:56.158424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eda6b7-2343-4a60-a8d4-a83b91d6ab83 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:56.167009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba3128-3bea-4d70-a021-60b7f5967102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:56.168656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba3128-3bea-4d70-a021-60b7f5967102 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.169135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba3128-3bea-4d70-a021-60b7f5967102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.169505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba3128-3bea-4d70-a021-60b7f5967102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.170592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba3128-3bea-4d70-a021-60b7f5967102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:56.175386 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba3128-3bea-4d70-a021-60b7f5967102 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.186111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1169/2342] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:40:56.190764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d73cad-2fe9-460c-9f07-2266efd2ac54 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:56.202322 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d73cad-2fe9-460c-9f07-2266efd2ac54 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:56.207671 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1587bfb-433a-4c31-adf8-f0afdaadfbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:56.208799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1587bfb-433a-4c31-adf8-f0afdaadfbd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.209311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1587bfb-433a-4c31-adf8-f0afdaadfbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.209962 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1587bfb-433a-4c31-adf8-f0afdaadfbd8 admin admin] PATH_INFO: `/v3/projects/7c3a677ae42a49538832ba7e24489ff8/users/d1237c9f979541feb603ee86b9cb3536/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.227316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1587bfb-433a-4c31-adf8-f0afdaadfbd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c3a677ae42a49538832ba7e24489ff8, user_id=d1237c9f979541feb603ee86b9cb3536, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:56.230157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1587bfb-433a-4c31-adf8-f0afdaadfbd8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.255036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1174/2343] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:40:56 2022] PUT /identity/v3/projects/7c3a677ae42a49538832ba7e24489ff8/users/d1237c9f979541feb603ee86b9cb3536/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:40:56.258949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba3128-3bea-4d70-a021-60b7f5967102 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:56.270811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba3128-3bea-4d70-a021-60b7f5967102 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:56.365809 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17f7ff-63e4-49e0-bb2c-7b2a510588e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:56.366700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17f7ff-63e4-49e0-bb2c-7b2a510588e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.367016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17f7ff-63e4-49e0-bb2c-7b2a510588e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.367430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17f7ff-63e4-49e0-bb2c-7b2a510588e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.370777 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17f7ff-63e4-49e0-bb2c-7b2a510588e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:56.373014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17f7ff-63e4-49e0-bb2c-7b2a510588e7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.387055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1170/2344] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:40:56.391146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1587bfb-433a-4c31-adf8-f0afdaadfbd8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:56.402403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1587bfb-433a-4c31-adf8-f0afdaadfbd8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:56.407131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d9eb3-bc48-4a86-b14e-fe0073dc5e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:56.407786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d9eb3-bc48-4a86-b14e-fe0073dc5e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:56.408074 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d9eb3-bc48-4a86-b14e-fe0073dc5e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:56.408499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d9eb3-bc48-4a86-b14e-fe0073dc5e49 admin admin] PATH_INFO: `/v3/projects/7c3a677ae42a49538832ba7e24489ff8/users/d1237c9f979541feb603ee86b9cb3536/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:56.410758 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d9eb3-bc48-4a86-b14e-fe0073dc5e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c3a677ae42a49538832ba7e24489ff8, user_id=d1237c9f979541feb603ee86b9cb3536, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:56.416667 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d9eb3-bc48-4a86-b14e-fe0073dc5e49 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:56.429721 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1175/2345] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:40:56 2022] PUT /identity/v3/projects/7c3a677ae42a49538832ba7e24489ff8/users/d1237c9f979541feb603ee86b9cb3536/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:40:58.050119 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17f7ff-63e4-49e0-bb2c-7b2a510588e7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:40:58.053319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ea17f7ff-63e4-49e0-bb2c-7b2a510588e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:58.073833 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:40:58.120956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17f7ff-63e4-49e0-bb2c-7b2a510588e7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:40:58.219143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9301c-1ba9-42b8-9c0f-1c91807990c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:40:58.220303 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9301c-1ba9-42b8-9c0f-1c91807990c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:40:58.220593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9301c-1ba9-42b8-9c0f-1c91807990c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:40:58.220928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9301c-1ba9-42b8-9c0f-1c91807990c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:40:58.226962 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1af9301c-1ba9-42b8-9c0f-1c91807990c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:40:58.331481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9301c-1ba9-42b8-9c0f-1c91807990c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:40:58.333184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9301c-1ba9-42b8-9c0f-1c91807990c3 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:40:58.345402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1171/2346] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:40:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:41:01.988086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eb6b4-54bc-413c-a71b-4cd9784e901a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:01.988993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eb6b4-54bc-413c-a71b-4cd9784e901a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:01.989426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eb6b4-54bc-413c-a71b-4cd9784e901a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:02.041231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e86eb6b4-54bc-413c-a71b-4cd9784e901a None None] MFA Rules not processed for user `d1237c9f979541feb603ee86b9cb3536`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:02.044920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e86eb6b4-54bc-413c-a71b-4cd9784e901a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:02.061991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:02.169755 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1176/2347] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:41:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:41:02.184264 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9301c-1ba9-42b8-9c0f-1c91807990c3 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:02.185761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1af9301c-1ba9-42b8-9c0f-1c91807990c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:02.371752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9301c-1ba9-42b8-9c0f-1c91807990c3 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:02.385515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558b028-d40f-4948-9258-429c4def0d5e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f57fa108c3d478a9976f4aea5f33bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:02.386103 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558b028-d40f-4948-9258-429c4def0d5e service glance] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:02.386495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558b028-d40f-4948-9258-429c4def0d5e service glance] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:02.387047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558b028-d40f-4948-9258-429c4def0d5e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:02.435469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:02.482098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558b028-d40f-4948-9258-429c4def0d5e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:02.488590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558b028-d40f-4948-9258-429c4def0d5e service glance] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:02.504050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1172/2348] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:41:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:41:02.675173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822b6ef-dfc8-4729-8350-4cd6e4afce36 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:02.675464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822b6ef-dfc8-4729-8350-4cd6e4afce36 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:02.675743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822b6ef-dfc8-4729-8350-4cd6e4afce36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:02.696477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c822b6ef-dfc8-4729-8350-4cd6e4afce36 None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:02.697723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c822b6ef-dfc8-4729-8350-4cd6e4afce36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:02.764573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1177/2349] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:41:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:41:02.777634 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558b028-d40f-4948-9258-429c4def0d5e service glance] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:02.779104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4558b028-d40f-4948-9258-429c4def0d5e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:02.856060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558b028-d40f-4948-9258-429c4def0d5e service glance] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:02.861014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8711335-fdce-42ea-b838-03ad52360a6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:02.861705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8711335-fdce-42ea-b838-03ad52360a6c service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:02.862171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8711335-fdce-42ea-b838-03ad52360a6c service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:02.862454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8711335-fdce-42ea-b838-03ad52360a6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:02.871362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8711335-fdce-42ea-b838-03ad52360a6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:02.872995 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8711335-fdce-42ea-b838-03ad52360a6c service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:02.887404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1173/2350] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:41:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:41:11.910999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4a3be-92cc-41c2-b4a4-a06147dc6422 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:11.911495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4a3be-92cc-41c2-b4a4-a06147dc6422 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:11.911610 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4a3be-92cc-41c2-b4a4-a06147dc6422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:11.930129 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-10a4a3be-92cc-41c2-b4a4-a06147dc6422 None None] MFA Rules not processed for user `6e461e65817648b0a878d8280c84dc6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:11.931348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-10a4a3be-92cc-41c2-b4a4-a06147dc6422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:11.938081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1178/2351] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:41:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:41:12.879664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=43 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:16.435743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4143b-a2d2-4fdd-b1ca-7b9626a16526 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:16.436290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4143b-a2d2-4fdd-b1ca-7b9626a16526 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:16.436290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4143b-a2d2-4fdd-b1ca-7b9626a16526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:16.456076 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-99c4143b-a2d2-4fdd-b1ca-7b9626a16526 None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:16.457315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-99c4143b-a2d2-4fdd-b1ca-7b9626a16526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:16.464559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1174/2352] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:41:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:41:16.477446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4a3be-92cc-41c2-b4a4-a06147dc6422 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:16.490058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4a3be-92cc-41c2-b4a4-a06147dc6422 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:16.495120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c133eb2-ef8c-43c9-a2f3-8e10e5a42571 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:16.495745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c133eb2-ef8c-43c9-a2f3-8e10e5a42571 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:16.496033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c133eb2-ef8c-43c9-a2f3-8e10e5a42571 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:16.496313 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c133eb2-ef8c-43c9-a2f3-8e10e5a42571 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:16.505242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c133eb2-ef8c-43c9-a2f3-8e10e5a42571 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:16.506827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c133eb2-ef8c-43c9-a2f3-8e10e5a42571 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:16.515059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1179/2353] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:41:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:41:26.450648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:26.512287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=61 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:45.000327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd08d0-8ef0-430a-83ba-d1bfdba7879b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:45.000868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd08d0-8ef0-430a-83ba-d1bfdba7879b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:45.001079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd08d0-8ef0-430a-83ba-d1bfdba7879b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:45.021833 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-36dd08d0-8ef0-430a-83ba-d1bfdba7879b None None] MFA Rules not processed for user `42a2bc662fa4405791532ebf2bce909c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:45.023038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-36dd08d0-8ef0-430a-83ba-d1bfdba7879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:45.080134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1175/2354] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:41:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:41:54.953990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c133eb2-ef8c-43c9-a2f3-8e10e5a42571 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:41:54.965241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c133eb2-ef8c-43c9-a2f3-8e10e5a42571 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:41:54.969176 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7385e-cd32-4c88-8bf9-d2a45952d32d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:41:54.969799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7385e-cd32-4c88-8bf9-d2a45952d32d service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:54.970085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7385e-cd32-4c88-8bf9-d2a45952d32d service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:54.971473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7385e-cd32-4c88-8bf9-d2a45952d32d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:54.981029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7385e-cd32-4c88-8bf9-d2a45952d32d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:41:54.982753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7385e-cd32-4c88-8bf9-d2a45952d32d service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:41:54.993172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1180/2355] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:41:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:41:55.072965 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=25 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:41:58.679056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb8d0f-016a-4cee-880b-a4004dd05285 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:41:58.679795 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb8d0f-016a-4cee-880b-a4004dd05285 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:41:58.679795 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb8d0f-016a-4cee-880b-a4004dd05285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:41:58.701469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e5fb8d0f-016a-4cee-880b-a4004dd05285 None None] MFA Rules not processed for user `1e02f414ef2a4990bd1492a63d074319`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:41:58.702712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e5fb8d0f-016a-4cee-880b-a4004dd05285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:41:58.780819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1176/2356] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:41:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:42:04.988705 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:07.748303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7385e-cd32-4c88-8bf9-d2a45952d32d service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:07.758559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7385e-cd32-4c88-8bf9-d2a45952d32d service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:07.762125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6aa62b-b7d0-455d-b297-56d4075f6bc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:07.762725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aa62b-b7d0-455d-b297-56d4075f6bc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:07.763006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aa62b-b7d0-455d-b297-56d4075f6bc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:07.763310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aa62b-b7d0-455d-b297-56d4075f6bc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:07.765060 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cd6aa62b-b7d0-455d-b297-56d4075f6bc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:07.832671 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6aa62b-b7d0-455d-b297-56d4075f6bc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:07.834093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6aa62b-b7d0-455d-b297-56d4075f6bc7 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:07.844617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1181/2357] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:42:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:42:08.772518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=25 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:09.021044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb8d0f-016a-4cee-880b-a4004dd05285 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:09.030545 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb8d0f-016a-4cee-880b-a4004dd05285 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:09.034149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebe089-ff3f-46e0-933c-747e616a2424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:09.034755 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebe089-ff3f-46e0-933c-747e616a2424 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:09.035040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebe089-ff3f-46e0-933c-747e616a2424 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:09.035356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebe089-ff3f-46e0-933c-747e616a2424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:09.037107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-05ebe089-ff3f-46e0-933c-747e616a2424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:09.050850 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebe089-ff3f-46e0-933c-747e616a2424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:09.052382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebe089-ff3f-46e0-933c-747e616a2424 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:09.063254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1177/2358] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:42:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:42:12.703606 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908449f7-a211-4993-836c-28332309e0b8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:12.704123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908449f7-a211-4993-836c-28332309e0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:12.704222 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908449f7-a211-4993-836c-28332309e0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:12.724189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-908449f7-a211-4993-836c-28332309e0b8 None None] MFA Rules not processed for user `db6ac4cf9dd84e338f944b4905dcdb50`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:12.725450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-908449f7-a211-4993-836c-28332309e0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:12.779484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1182/2359] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:42:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:42:19.058503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=6 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:22.772611 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=59 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:30.900002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebe089-ff3f-46e0-933c-747e616a2424 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:30.901428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-05ebe089-ff3f-46e0-933c-747e616a2424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:30.958326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebe089-ff3f-46e0-933c-747e616a2424 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:30.961962 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158c40c-0618-46a6-99fa-d79e399af1e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:30.962577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158c40c-0618-46a6-99fa-d79e399af1e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:30.962860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158c40c-0618-46a6-99fa-d79e399af1e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:30.963184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158c40c-0618-46a6-99fa-d79e399af1e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:30.964959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9158c40c-0618-46a6-99fa-d79e399af1e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:31.024836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158c40c-0618-46a6-99fa-d79e399af1e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:31.026284 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158c40c-0618-46a6-99fa-d79e399af1e6 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:31.035422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1178/2360] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:42:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:42:38.140978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e68dfa-bd6e-47d8-8e59-7ea500e2831b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:38.141484 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e68dfa-bd6e-47d8-8e59-7ea500e2831b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:38.141587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e68dfa-bd6e-47d8-8e59-7ea500e2831b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:38.161237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b3e68dfa-bd6e-47d8-8e59-7ea500e2831b None None] MFA Rules not processed for user `5958b9f3a2c4424db419e7b1c667ca92`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:38.162449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b3e68dfa-bd6e-47d8-8e59-7ea500e2831b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:38.169303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1183/2361] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:42:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:42:38.475114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158c40c-0618-46a6-99fa-d79e399af1e6 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:38.484345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158c40c-0618-46a6-99fa-d79e399af1e6 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:38.487962 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb3f0b-be20-4b1f-b8d3-318b8ea78f48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:38.488601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb3f0b-be20-4b1f-b8d3-318b8ea78f48 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:38.488909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb3f0b-be20-4b1f-b8d3-318b8ea78f48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:38.489169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb3f0b-be20-4b1f-b8d3-318b8ea78f48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:38.490899 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5bfb3f0b-be20-4b1f-b8d3-318b8ea78f48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:38.506819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb3f0b-be20-4b1f-b8d3-318b8ea78f48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:38.506819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb3f0b-be20-4b1f-b8d3-318b8ea78f48 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:38.512944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1179/2362] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:42:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:42:38.828305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e68dfa-bd6e-47d8-8e59-7ea500e2831b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:38.829870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b3e68dfa-bd6e-47d8-8e59-7ea500e2831b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:38.912819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e68dfa-bd6e-47d8-8e59-7ea500e2831b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:38.918036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1e0ba-0d9a-4c6e-9484-3655ed2ef73d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928c5d1284b74f1291c031323bb03a20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:38.918778 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1e0ba-0d9a-4c6e-9484-3655ed2ef73d service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:38.919085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1e0ba-0d9a-4c6e-9484-3655ed2ef73d service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:38.919375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1e0ba-0d9a-4c6e-9484-3655ed2ef73d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:38.921197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b0a1e0ba-0d9a-4c6e-9484-3655ed2ef73d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:38.934422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1e0ba-0d9a-4c6e-9484-3655ed2ef73d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:38.936082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1e0ba-0d9a-4c6e-9484-3655ed2ef73d service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:38.947670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1184/2363] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:42:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:42:42.942991 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800d323-0e8f-47e6-ade0-571e6842f454 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:42.943618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800d323-0e8f-47e6-ade0-571e6842f454 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:42.943618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800d323-0e8f-47e6-ade0-571e6842f454 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:42.945059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1180/2364] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:42.958227 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9fb44-ebaf-402a-8567-76c13d163480 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:42.959322 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9fb44-ebaf-402a-8567-76c13d163480 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:42.959626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9fb44-ebaf-402a-8567-76c13d163480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:42.985021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f0f9fb44-ebaf-402a-8567-76c13d163480 None None] MFA Rules not processed for user `138ede2854894a3b995435980965911c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:42.987247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f0f9fb44-ebaf-402a-8567-76c13d163480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:42.997585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1185/2365] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:42:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:42:43.018690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800d323-0e8f-47e6-ade0-571e6842f454 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:43.032902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800d323-0e8f-47e6-ade0-571e6842f454 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:43.038326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7f645-43ae-40df-ade3-eed9e4d73115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:43.039871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7f645-43ae-40df-ade3-eed9e4d73115 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.040158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7f645-43ae-40df-ade3-eed9e4d73115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.041003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7f645-43ae-40df-ade3-eed9e4d73115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.050717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7f645-43ae-40df-ade3-eed9e4d73115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:43.052735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7f645-43ae-40df-ade3-eed9e4d73115 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:43.061542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1181/2366] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:42:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:42:43.184200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bf0e5-c3b6-4772-8b16-8e78e7fecd64 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.184505 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bf0e5-c3b6-4772-8b16-8e78e7fecd64 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.184769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bf0e5-c3b6-4772-8b16-8e78e7fecd64 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.186121 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1186/2367] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:42:43.191505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4a83a-e768-43d7-ad0b-7072a940d845 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.192605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4a83a-e768-43d7-ad0b-7072a940d845 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.192963 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4a83a-e768-43d7-ad0b-7072a940d845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.215344 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-4bd4a83a-e768-43d7-ad0b-7072a940d845 None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:42:43.216575 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4bd4a83a-e768-43d7-ad0b-7072a940d845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:43.229544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1182/2368] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:42:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:42:43.252046 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89bf0e5-c3b6-4772-8b16-8e78e7fecd64 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:43.268283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89bf0e5-c3b6-4772-8b16-8e78e7fecd64 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:43.274709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e86fd-9f2b-4af8-b1b7-34ebe8f70a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:43.275435 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e86fd-9f2b-4af8-b1b7-34ebe8f70a6f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:43.275860 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e86fd-9f2b-4af8-b1b7-34ebe8f70a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:43.276327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e86fd-9f2b-4af8-b1b7-34ebe8f70a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:43.288283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e86fd-9f2b-4af8-b1b7-34ebe8f70a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:43.289722 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e86fd-9f2b-4af8-b1b7-34ebe8f70a6f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:43.302016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1187/2369] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:42:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:42:49.565319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4a83a-e768-43d7-ad0b-7072a940d845 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:49.574826 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4a83a-e768-43d7-ad0b-7072a940d845 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:49.578686 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db3ca0-e152-4573-a669-17f5ffa5c80a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:49.579320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db3ca0-e152-4573-a669-17f5ffa5c80a service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:49.579602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db3ca0-e152-4573-a669-17f5ffa5c80a service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:49.579870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db3ca0-e152-4573-a669-17f5ffa5c80a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:49.582153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c4db3ca0-e152-4573-a669-17f5ffa5c80a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:49.591472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db3ca0-e152-4573-a669-17f5ffa5c80a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:49.592885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db3ca0-e152-4573-a669-17f5ffa5c80a service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:49.601429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1183/2370] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:42:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:42:50.194681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e86fd-9f2b-4af8-b1b7-34ebe8f70a6f service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:50.196306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a72e86fd-9f2b-4af8-b1b7-34ebe8f70a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:50.264309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e86fd-9f2b-4af8-b1b7-34ebe8f70a6f service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:50.268958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:50.269627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.270028 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.270317 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] PATH_INFO: `/v3/users/42a2bc662fa4405791532ebf2bce909c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.271518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42a2bc662fa4405791532ebf2bce909c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:50.273444 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:50.319136 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42a2bc662fa4405791532ebf2bce909c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:42:50.326254 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42a2bc662fa4405791532ebf2bce909c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:42:50.334115 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:50.339177 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42a2bc662fa4405791532ebf2bce909c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:42:50.339911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42a2bc662fa4405791532ebf2bce909c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:42:50.341205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42a2bc662fa4405791532ebf2bce909c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:42:50.355872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1188/2371] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:42:50 2022] DELETE /identity/v3/users/42a2bc662fa4405791532ebf2bce909c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:42:50.623796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db3ca0-e152-4573-a669-17f5ffa5c80a service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:50.636176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db3ca0-e152-4573-a669-17f5ffa5c80a service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:50.699990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:50.700641 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:50.700927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:50.701204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] PATH_INFO: `/v3/projects/4a76209de52146bf8440fddcdd00b284` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:50.702370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a76209de52146bf8440fddcdd00b284)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:50.704373 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:50.716226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:42:50.752087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a76209de52146bf8440fddcdd00b284', 'request_id': 'req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:42:50.759481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a76209de52146bf8440fddcdd00b284', 'request_id': 'req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:42:50.765637 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a76209de52146bf8440fddcdd00b284', 'request_id': 'req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:42:50.777597 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] The token cache is being invalidate because project 4a76209de52146bf8440fddcdd00b284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:42:50.777885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:42:50.778494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:42:50.780149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1184/2372] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:42:50 2022] DELETE /identity/v3/projects/4a76209de52146bf8440fddcdd00b284 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:42:53.073316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:42:53.074831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:53.149397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9fbef-bd45-4463-968d-bbf9a2c0b665 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:42:53.214005 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8b2d6-bf8a-4b1f-8a92-a4d0e7967d96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:42:53.214540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8b2d6-bf8a-4b1f-8a92-a4d0e7967d96 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:42:53.214830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8b2d6-bf8a-4b1f-8a92-a4d0e7967d96 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:42:53.215139 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8b2d6-bf8a-4b1f-8a92-a4d0e7967d96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:42:53.216967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6ad8b2d6-bf8a-4b1f-8a92-a4d0e7967d96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:42:53.230526 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8b2d6-bf8a-4b1f-8a92-a4d0e7967d96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:42:53.232187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8b2d6-bf8a-4b1f-8a92-a4d0e7967d96 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:42:53.243627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1189/2373] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:42:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:00.774595 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:03.239076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=69,keystone:DELETE=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:03.614381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:03.615817 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:03.674325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e64407-7e9a-4970-a1e1-6e8f2aa88d46 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:03.678247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae42a36-fb3f-452d-b17e-cc0d76138f6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:03.678909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae42a36-fb3f-452d-b17e-cc0d76138f6c service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:03.679231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae42a36-fb3f-452d-b17e-cc0d76138f6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:03.679509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae42a36-fb3f-452d-b17e-cc0d76138f6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:03.681247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cae42a36-fb3f-452d-b17e-cc0d76138f6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:03.742486 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae42a36-fb3f-452d-b17e-cc0d76138f6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:03.743958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae42a36-fb3f-452d-b17e-cc0d76138f6c service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:03.753186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1185/2374] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:06.417384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8b2d6-bf8a-4b1f-8a92-a4d0e7967d96 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:06.418707 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6ad8b2d6-bf8a-4b1f-8a92-a4d0e7967d96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:06.433152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8b2d6-bf8a-4b1f-8a92-a4d0e7967d96 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:06.439145 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b31d00-3721-46da-aedf-d184a804fc4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:06.439749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b31d00-3721-46da-aedf-d184a804fc4f service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:06.440029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b31d00-3721-46da-aedf-d184a804fc4f service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:06.440303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b31d00-3721-46da-aedf-d184a804fc4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:06.442089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f4b31d00-3721-46da-aedf-d184a804fc4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:06.508906 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b31d00-3721-46da-aedf-d184a804fc4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:06.510517 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b31d00-3721-46da-aedf-d184a804fc4f service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:06.519959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1190/2375] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:13.748800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=48 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:16.515502 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=28 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:22.598920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae42a36-fb3f-452d-b17e-cc0d76138f6c service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:22.600393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cae42a36-fb3f-452d-b17e-cc0d76138f6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:22.656353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae42a36-fb3f-452d-b17e-cc0d76138f6c service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:22.659955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1be973-e1f9-454d-8cc3-d080baf0b5ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:22.660662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1be973-e1f9-454d-8cc3-d080baf0b5ef service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:22.660948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1be973-e1f9-454d-8cc3-d080baf0b5ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:22.661225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1be973-e1f9-454d-8cc3-d080baf0b5ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:22.662942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8d1be973-e1f9-454d-8cc3-d080baf0b5ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:22.725801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1be973-e1f9-454d-8cc3-d080baf0b5ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:22.727269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1be973-e1f9-454d-8cc3-d080baf0b5ef service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:22.736831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1186/2376] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:26.433127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd559eb9-bbfb-4b6d-85d1-2781e38c30a3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:26.433674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd559eb9-bbfb-4b6d-85d1-2781e38c30a3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:26.433674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd559eb9-bbfb-4b6d-85d1-2781e38c30a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:26.453155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bd559eb9-bbfb-4b6d-85d1-2781e38c30a3 None None] MFA Rules not processed for user `890be2e19b43473caa79b2dc889f8436`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:26.454389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd559eb9-bbfb-4b6d-85d1-2781e38c30a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:26.525541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1191/2377] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:26.545346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1be973-e1f9-454d-8cc3-d080baf0b5ef service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:26.546874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8d1be973-e1f9-454d-8cc3-d080baf0b5ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:26.595021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd559eb9-bbfb-4b6d-85d1-2781e38c30a3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:26.596728 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd559eb9-bbfb-4b6d-85d1-2781e38c30a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:26.615655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd559eb9-bbfb-4b6d-85d1-2781e38c30a3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:26.627621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98536ca-92a0-48ea-9fe6-83aabb074f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:26.628580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98536ca-92a0-48ea-9fe6-83aabb074f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:26.628901 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98536ca-92a0-48ea-9fe6-83aabb074f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:26.629186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98536ca-92a0-48ea-9fe6-83aabb074f74 admin admin] PATH_INFO: `/v3/users/db6ac4cf9dd84e338f944b4905dcdb50` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:26.630407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98536ca-92a0-48ea-9fe6-83aabb074f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db6ac4cf9dd84e338f944b4905dcdb50)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:26.632555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98536ca-92a0-48ea-9fe6-83aabb074f74 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:26.639601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1be973-e1f9-454d-8cc3-d080baf0b5ef service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:26.645141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31539ff-c6ce-4175-b0df-1e8a5204211b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928c5d1284b74f1291c031323bb03a20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:26.645967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31539ff-c6ce-4175-b0df-1e8a5204211b service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:26.646419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31539ff-c6ce-4175-b0df-1e8a5204211b service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:26.646698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31539ff-c6ce-4175-b0df-1e8a5204211b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:26.648723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c31539ff-c6ce-4175-b0df-1e8a5204211b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:26.663054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31539ff-c6ce-4175-b0df-1e8a5204211b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:26.665024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31539ff-c6ce-4175-b0df-1e8a5204211b service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:26.682086 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1187/2378] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:26.690307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b98536ca-92a0-48ea-9fe6-83aabb074f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db6ac4cf9dd84e338f944b4905dcdb50'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:26.700609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b98536ca-92a0-48ea-9fe6-83aabb074f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db6ac4cf9dd84e338f944b4905dcdb50'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:26.713717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b98536ca-92a0-48ea-9fe6-83aabb074f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db6ac4cf9dd84e338f944b4905dcdb50'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:26.714729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b98536ca-92a0-48ea-9fe6-83aabb074f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db6ac4cf9dd84e338f944b4905dcdb50'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:26.715450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b98536ca-92a0-48ea-9fe6-83aabb074f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db6ac4cf9dd84e338f944b4905dcdb50'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:26.729197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1192/2379] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:43:26 2022] DELETE /identity/v3/users/db6ac4cf9dd84e338f944b4905dcdb50 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:43:26.913151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31539ff-c6ce-4175-b0df-1e8a5204211b service placement] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:26.927309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31539ff-c6ce-4175-b0df-1e8a5204211b service placement] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:26.975666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:26.995911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a697b-ee57-4a86-b40c-63f3221f2b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:26.996564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a697b-ee57-4a86-b40c-63f3221f2b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:26.996855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a697b-ee57-4a86-b40c-63f3221f2b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:26.997139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a697b-ee57-4a86-b40c-63f3221f2b58 admin admin] PATH_INFO: `/v3/projects/015cc91d3e2f4d3791301c05e4c8be20` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:26.998313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a697b-ee57-4a86-b40c-63f3221f2b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=015cc91d3e2f4d3791301c05e4c8be20)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:27.000263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a697b-ee57-4a86-b40c-63f3221f2b58 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:27.048886 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-af6a697b-ee57-4a86-b40c-63f3221f2b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '015cc91d3e2f4d3791301c05e4c8be20', 'request_id': 'req-af6a697b-ee57-4a86-b40c-63f3221f2b58'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:27.054853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-af6a697b-ee57-4a86-b40c-63f3221f2b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '015cc91d3e2f4d3791301c05e4c8be20', 'request_id': 'req-af6a697b-ee57-4a86-b40c-63f3221f2b58'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:27.061055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-af6a697b-ee57-4a86-b40c-63f3221f2b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '015cc91d3e2f4d3791301c05e4c8be20', 'request_id': 'req-af6a697b-ee57-4a86-b40c-63f3221f2b58'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:27.075019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-af6a697b-ee57-4a86-b40c-63f3221f2b58 admin admin] The token cache is being invalidate because project 015cc91d3e2f4d3791301c05e4c8be20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:43:27.075918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-af6a697b-ee57-4a86-b40c-63f3221f2b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:27.076099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-af6a697b-ee57-4a86-b40c-63f3221f2b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:27.077656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1188/2380] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:43:26 2022] DELETE /identity/v3/projects/015cc91d3e2f4d3791301c05e4c8be20 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:27.083391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb935283-3e38-42af-86b7-d7c2bcbd8ef0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.084049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb935283-3e38-42af-86b7-d7c2bcbd8ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.084267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb935283-3e38-42af-86b7-d7c2bcbd8ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.104843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bb935283-3e38-42af-86b7-d7c2bcbd8ef0 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:27.106112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bb935283-3e38-42af-86b7-d7c2bcbd8ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:27.168021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1193/2381] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:27.176735 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e901ec-01aa-4258-8b7d-2ee10cdbe861 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.176989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e901ec-01aa-4258-8b7d-2ee10cdbe861 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.177297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e901ec-01aa-4258-8b7d-2ee10cdbe861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.197981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c9e901ec-01aa-4258-8b7d-2ee10cdbe861 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:27.199234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c9e901ec-01aa-4258-8b7d-2ee10cdbe861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:27.207700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1189/2382] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:27.211956 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb935283-3e38-42af-86b7-d7c2bcbd8ef0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:27.223229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb935283-3e38-42af-86b7-d7c2bcbd8ef0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:27.229009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70239ff-7792-4b13-8340-6735b841aa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:27.229636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70239ff-7792-4b13-8340-6735b841aa3f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.230005 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70239ff-7792-4b13-8340-6735b841aa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.230298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70239ff-7792-4b13-8340-6735b841aa3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.230996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70239ff-7792-4b13-8340-6735b841aa3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:27.231337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70239ff-7792-4b13-8340-6735b841aa3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:27.232940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70239ff-7792-4b13-8340-6735b841aa3f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:27.244788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1194/2383] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:27.249308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e901ec-01aa-4258-8b7d-2ee10cdbe861 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:27.261761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e901ec-01aa-4258-8b7d-2ee10cdbe861 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:27.266787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a24301-f1b0-4e52-8773-6899b212427e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:27.267539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a24301-f1b0-4e52-8773-6899b212427e admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.267712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a24301-f1b0-4e52-8773-6899b212427e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.268156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a24301-f1b0-4e52-8773-6899b212427e admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.269075 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a24301-f1b0-4e52-8773-6899b212427e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:27.272191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a24301-f1b0-4e52-8773-6899b212427e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:27.320506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1190/2384] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:43:27 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:27.322393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70239ff-7792-4b13-8340-6735b841aa3f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:27.330213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:27.337534 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70239ff-7792-4b13-8340-6735b841aa3f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:27.422326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336381da-65be-45ed-9750-0834a098f911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:27.423302 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336381da-65be-45ed-9750-0834a098f911 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.423711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336381da-65be-45ed-9750-0834a098f911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.424057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336381da-65be-45ed-9750-0834a098f911 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.424986 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336381da-65be-45ed-9750-0834a098f911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:27.427272 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336381da-65be-45ed-9750-0834a098f911 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:27.458356 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1195/2385] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:43:27 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:43:27.459658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a24301-f1b0-4e52-8773-6899b212427e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:27.476834 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a24301-f1b0-4e52-8773-6899b212427e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:27.490428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1d153-f022-4165-8c6b-87630cb66331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:27.491257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1d153-f022-4165-8c6b-87630cb66331 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.491616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1d153-f022-4165-8c6b-87630cb66331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.491906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1d153-f022-4165-8c6b-87630cb66331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.492612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1d153-f022-4165-8c6b-87630cb66331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:27.494375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1d153-f022-4165-8c6b-87630cb66331 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:27.506413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1191/2386] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:27.510339 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336381da-65be-45ed-9750-0834a098f911 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:27.521319 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336381da-65be-45ed-9750-0834a098f911 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:27.527162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb37483-5424-4669-bcde-b68f68e22415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:27.527969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb37483-5424-4669-bcde-b68f68e22415 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.528427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb37483-5424-4669-bcde-b68f68e22415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.528895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb37483-5424-4669-bcde-b68f68e22415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.529740 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb37483-5424-4669-bcde-b68f68e22415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:27.531376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb37483-5424-4669-bcde-b68f68e22415 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:27.548881 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1196/2387] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:27.550186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1d153-f022-4165-8c6b-87630cb66331 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:27.560980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1d153-f022-4165-8c6b-87630cb66331 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:27.566390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9dbec-89c2-404c-a8ff-4d4dff406d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:27.567297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9dbec-89c2-404c-a8ff-4d4dff406d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.567585 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9dbec-89c2-404c-a8ff-4d4dff406d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.568167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9dbec-89c2-404c-a8ff-4d4dff406d9a admin admin] PATH_INFO: `/v3/projects/698542bb944943809d891c35e7e5327a/users/a34b98469a3244f287772ef8c50fb8e8/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.587739 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9dbec-89c2-404c-a8ff-4d4dff406d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=698542bb944943809d891c35e7e5327a, user_id=a34b98469a3244f287772ef8c50fb8e8, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:27.591287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9dbec-89c2-404c-a8ff-4d4dff406d9a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:27.608089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1192/2388] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:43:27 2022] PUT /identity/v3/projects/698542bb944943809d891c35e7e5327a/users/a34b98469a3244f287772ef8c50fb8e8/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:27.620897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f25d5-eb25-41bb-8fe7-52c2af3460c2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.621734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f25d5-eb25-41bb-8fe7-52c2af3460c2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.622942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f25d5-eb25-41bb-8fe7-52c2af3460c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.658857 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0d7f25d5-eb25-41bb-8fe7-52c2af3460c2 None None] MFA Rules not processed for user `a34b98469a3244f287772ef8c50fb8e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:27.660714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0d7f25d5-eb25-41bb-8fe7-52c2af3460c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:27.741535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1197/2389] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:27.752962 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9dbec-89c2-404c-a8ff-4d4dff406d9a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:27.756018 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-efb9dbec-89c2-404c-a8ff-4d4dff406d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:27.846247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:27.851953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9dbec-89c2-404c-a8ff-4d4dff406d9a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:27.861822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7364e79-a76d-43dc-bf4d-a48e0c7f5e7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:27.862501 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7364e79-a76d-43dc-bf4d-a48e0c7f5e7d service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:27.862784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7364e79-a76d-43dc-bf4d-a48e0c7f5e7d service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:27.863243 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7364e79-a76d-43dc-bf4d-a48e0c7f5e7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:27.872294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7364e79-a76d-43dc-bf4d-a48e0c7f5e7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:27.873940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7364e79-a76d-43dc-bf4d-a48e0c7f5e7d service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:27.883426 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1193/2390] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:28.741943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ab4b5-30a8-4142-8cea-4ebedc16c618 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:28.742459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ab4b5-30a8-4142-8cea-4ebedc16c618 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:28.742573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ab4b5-30a8-4142-8cea-4ebedc16c618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:28.762817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-006ab4b5-30a8-4142-8cea-4ebedc16c618 None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:28.764095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-006ab4b5-30a8-4142-8cea-4ebedc16c618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:28.829044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1198/2391] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:43:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:28.874664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7364e79-a76d-43dc-bf4d-a48e0c7f5e7d service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:28.876515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a7364e79-a76d-43dc-bf4d-a48e0c7f5e7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:28.949731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7364e79-a76d-43dc-bf4d-a48e0c7f5e7d service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:28.956009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82678f-1892-45e3-a6e6-76f609966966 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:28.956643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82678f-1892-45e3-a6e6-76f609966966 service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:28.957132 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82678f-1892-45e3-a6e6-76f609966966 service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:28.957699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82678f-1892-45e3-a6e6-76f609966966 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:28.968994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82678f-1892-45e3-a6e6-76f609966966 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:28.971752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82678f-1892-45e3-a6e6-76f609966966 service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:28.982638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1194/2392] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:43:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:43:29.811516 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ab4b5-30a8-4142-8cea-4ebedc16c618 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:29.833573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ab4b5-30a8-4142-8cea-4ebedc16c618 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:29.850434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e102cf-ba2b-4164-b9f9-7cdb3abbf97c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:29.851525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e102cf-ba2b-4164-b9f9-7cdb3abbf97c service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:29.852033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e102cf-ba2b-4164-b9f9-7cdb3abbf97c service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:29.852550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e102cf-ba2b-4164-b9f9-7cdb3abbf97c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:29.857701 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a5e102cf-ba2b-4164-b9f9-7cdb3abbf97c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:29.889270 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e102cf-ba2b-4164-b9f9-7cdb3abbf97c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:29.892238 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e102cf-ba2b-4164-b9f9-7cdb3abbf97c service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:29.912419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1199/2393] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:30.873949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82678f-1892-45e3-a6e6-76f609966966 service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:30.957408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82678f-1892-45e3-a6e6-76f609966966 service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:30.968117 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:30.968825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:30.969107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:30.969579 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] PATH_INFO: `/v3/users/a34b98469a3244f287772ef8c50fb8e8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:30.971120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a34b98469a3244f287772ef8c50fb8e8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:30.973853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:31.032490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a34b98469a3244f287772ef8c50fb8e8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:31.049349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a34b98469a3244f287772ef8c50fb8e8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:31.051778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a34b98469a3244f287772ef8c50fb8e8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:31.059428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a34b98469a3244f287772ef8c50fb8e8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:31.070895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a34b98469a3244f287772ef8c50fb8e8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:31.073264 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:31.082355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1195/2394] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:43:30 2022] DELETE /identity/v3/users/a34b98469a3244f287772ef8c50fb8e8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:31.083717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37bbbf-1ef5-4dcc-ac77-64efe9a38e65 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:31.084583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37bbbf-1ef5-4dcc-ac77-64efe9a38e65 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:31.084865 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37bbbf-1ef5-4dcc-ac77-64efe9a38e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:31.093656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:31.134967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8f37bbbf-1ef5-4dcc-ac77-64efe9a38e65 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:31.136629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8f37bbbf-1ef5-4dcc-ac77-64efe9a38e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:31.229606 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1200/2395] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:31.244048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:31.246347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:31.407799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f6fcf-a1af-4fd7-bc5e-af9585204721 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:31.423382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9975b-36d3-4832-88a1-a9e8f5dec9ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:31.424209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9975b-36d3-4832-88a1-a9e8f5dec9ef service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:31.424504 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9975b-36d3-4832-88a1-a9e8f5dec9ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:31.424798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9975b-36d3-4832-88a1-a9e8f5dec9ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:31.450441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9975b-36d3-4832-88a1-a9e8f5dec9ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:31.454460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9975b-36d3-4832-88a1-a9e8f5dec9ef service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:31.468482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1196/2396] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:31.875414 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37bbbf-1ef5-4dcc-ac77-64efe9a38e65 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:31.892258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8f37bbbf-1ef5-4dcc-ac77-64efe9a38e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:31.938303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37bbbf-1ef5-4dcc-ac77-64efe9a38e65 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:31.969294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:31.969983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:31.970265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:31.970539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076 admin admin] PATH_INFO: `/v3/projects/698542bb944943809d891c35e7e5327a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:31.971991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=698542bb944943809d891c35e7e5327a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:31.973969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:32.066146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '698542bb944943809d891c35e7e5327a', 'request_id': 'req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:32.076168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '698542bb944943809d891c35e7e5327a', 'request_id': 'req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:32.084965 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '698542bb944943809d891c35e7e5327a', 'request_id': 'req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:32.097122 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076 admin admin] The token cache is being invalidate because project 698542bb944943809d891c35e7e5327a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:43:32.097673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:32.098520 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f9ce6e72-fd8d-4dd7-8467-6736c45bc076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:32.100266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1201/2397] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:43:31 2022] DELETE /identity/v3/projects/698542bb944943809d891c35e7e5327a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:32.105496 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0fcf0-a26b-4c7d-b9a1-2662db84662b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:32.106743 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0fcf0-a26b-4c7d-b9a1-2662db84662b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:32.107287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0fcf0-a26b-4c7d-b9a1-2662db84662b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:32.132801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7fc0fcf0-a26b-4c7d-b9a1-2662db84662b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:32.134113 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7fc0fcf0-a26b-4c7d-b9a1-2662db84662b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:32.199603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1197/2398] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:43:32.207417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ffb1b-7275-46b6-bbcc-2f6404a3369b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:32.208018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ffb1b-7275-46b6-bbcc-2f6404a3369b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:32.208560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ffb1b-7275-46b6-bbcc-2f6404a3369b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:32.232799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b70ffb1b-7275-46b6-bbcc-2f6404a3369b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:32.234018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b70ffb1b-7275-46b6-bbcc-2f6404a3369b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:32.245413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1202/2399] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:32.249851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0fcf0-a26b-4c7d-b9a1-2662db84662b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:32.261523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0fcf0-a26b-4c7d-b9a1-2662db84662b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:32.266345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db5f74-5400-472e-8bf6-c284b2d598c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:32.267037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db5f74-5400-472e-8bf6-c284b2d598c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:32.267421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db5f74-5400-472e-8bf6-c284b2d598c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:32.267767 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db5f74-5400-472e-8bf6-c284b2d598c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:32.268550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db5f74-5400-472e-8bf6-c284b2d598c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:32.268969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db5f74-5400-472e-8bf6-c284b2d598c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:32.270903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db5f74-5400-472e-8bf6-c284b2d598c3 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:32.284179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1198/2400] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:32.289948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ffb1b-7275-46b6-bbcc-2f6404a3369b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:32.301272 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ffb1b-7275-46b6-bbcc-2f6404a3369b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:32.306397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d23cdd-a54f-4d83-95ce-c11bd3247dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:32.307029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d23cdd-a54f-4d83-95ce-c11bd3247dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:32.307346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d23cdd-a54f-4d83-95ce-c11bd3247dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:32.307660 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d23cdd-a54f-4d83-95ce-c11bd3247dc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:32.308334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d23cdd-a54f-4d83-95ce-c11bd3247dc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:32.310598 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d23cdd-a54f-4d83-95ce-c11bd3247dc1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:32.351059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1203/2401] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:43:32 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:32.352319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db5f74-5400-472e-8bf6-c284b2d598c3 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:32.365017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db5f74-5400-472e-8bf6-c284b2d598c3 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:32.493353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505a47d-2ab0-4180-83a5-f85546bcba48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:32.493978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505a47d-2ab0-4180-83a5-f85546bcba48 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:32.494269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505a47d-2ab0-4180-83a5-f85546bcba48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:32.494559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505a47d-2ab0-4180-83a5-f85546bcba48 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:32.495285 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505a47d-2ab0-4180-83a5-f85546bcba48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:32.497271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505a47d-2ab0-4180-83a5-f85546bcba48 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:32.529423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1199/2402] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:43:32 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:32.533195 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d23cdd-a54f-4d83-95ce-c11bd3247dc1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:32.546321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d23cdd-a54f-4d83-95ce-c11bd3247dc1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:32.552816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05028d-5002-48d6-b112-57161d8b7610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:32.553449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05028d-5002-48d6-b112-57161d8b7610 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:32.553735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05028d-5002-48d6-b112-57161d8b7610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:32.554014 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05028d-5002-48d6-b112-57161d8b7610 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:32.554718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05028d-5002-48d6-b112-57161d8b7610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:32.556361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05028d-5002-48d6-b112-57161d8b7610 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:32.572367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1204/2403] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:32.576013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505a47d-2ab0-4180-83a5-f85546bcba48 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:32.591255 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505a47d-2ab0-4180-83a5-f85546bcba48 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:32.600194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13db377-720b-4b6a-a507-58f4465df903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:32.601364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13db377-720b-4b6a-a507-58f4465df903 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:32.601773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13db377-720b-4b6a-a507-58f4465df903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:32.602055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13db377-720b-4b6a-a507-58f4465df903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:32.603274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13db377-720b-4b6a-a507-58f4465df903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:32.605503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13db377-720b-4b6a-a507-58f4465df903 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:32.622020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1200/2404] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:32.626652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05028d-5002-48d6-b112-57161d8b7610 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:32.651364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05028d-5002-48d6-b112-57161d8b7610 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:32.657423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6a3d2-3539-4a05-a97f-9de5a2e49b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:32.658432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6a3d2-3539-4a05-a97f-9de5a2e49b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:32.658719 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6a3d2-3539-4a05-a97f-9de5a2e49b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:32.658998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6a3d2-3539-4a05-a97f-9de5a2e49b91 admin admin] PATH_INFO: `/v3/projects/615102fbb0db49579b3982f01ee52808/users/871403ff190b4b8682dd340fbd126cd0/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:32.683228 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6a3d2-3539-4a05-a97f-9de5a2e49b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615102fbb0db49579b3982f01ee52808, user_id=871403ff190b4b8682dd340fbd126cd0, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:32.686105 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6a3d2-3539-4a05-a97f-9de5a2e49b91 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:32.716091 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1205/2405] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:43:32 2022] PUT /identity/v3/projects/615102fbb0db49579b3982f01ee52808/users/871403ff190b4b8682dd340fbd126cd0/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:32.717725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a15e43-20eb-4735-af3a-37b7c36b3eb6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:32.718080 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a15e43-20eb-4735-af3a-37b7c36b3eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:32.718428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a15e43-20eb-4735-af3a-37b7c36b3eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:32.750704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:32.771191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-58a15e43-20eb-4735-af3a-37b7c36b3eb6 None None] MFA Rules not processed for user `871403ff190b4b8682dd340fbd126cd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:32.771430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-58a15e43-20eb-4735-af3a-37b7c36b3eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:32.855655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1201/2406] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:32.857093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6a3d2-3539-4a05-a97f-9de5a2e49b91 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:32.896742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:32.940537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6a3d2-3539-4a05-a97f-9de5a2e49b91 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:32.950859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c092da-e4e5-414d-bf7c-b9e743295ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:32.951527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c092da-e4e5-414d-bf7c-b9e743295ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:32.951820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c092da-e4e5-414d-bf7c-b9e743295ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:32.952101 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c092da-e4e5-414d-bf7c-b9e743295ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:32.952838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c092da-e4e5-414d-bf7c-b9e743295ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:32.954822 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c092da-e4e5-414d-bf7c-b9e743295ea1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:32.995622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1206/2407] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:43:32 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:32.999120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a15e43-20eb-4735-af3a-37b7c36b3eb6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:33.010881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a15e43-20eb-4735-af3a-37b7c36b3eb6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:33.085603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb8fad-af9a-455b-b3d4-06d33d611320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:33.086229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8fad-af9a-455b-b3d4-06d33d611320 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.086516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8fad-af9a-455b-b3d4-06d33d611320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.086794 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8fad-af9a-455b-b3d4-06d33d611320 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.087653 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8fad-af9a-455b-b3d4-06d33d611320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:33.089883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8fad-af9a-455b-b3d4-06d33d611320 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:33.121618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1202/2408] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:43:32 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:43:33.124248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c092da-e4e5-414d-bf7c-b9e743295ea1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:33.137000 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c092da-e4e5-414d-bf7c-b9e743295ea1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:33.145916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48f852-5a31-467c-adce-0e199147722b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:33.146633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f852-5a31-467c-adce-0e199147722b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.146981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f852-5a31-467c-adce-0e199147722b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.147542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f852-5a31-467c-adce-0e199147722b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.148309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48f852-5a31-467c-adce-0e199147722b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:33.150019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48f852-5a31-467c-adce-0e199147722b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:33.165416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1207/2409] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:33.169768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb8fad-af9a-455b-b3d4-06d33d611320 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:33.183017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb8fad-af9a-455b-b3d4-06d33d611320 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:33.188550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dd2e2-8bcb-47bf-a255-0e8101e558e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:33.189226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd2e2-8bcb-47bf-a255-0e8101e558e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.189510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd2e2-8bcb-47bf-a255-0e8101e558e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.189799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd2e2-8bcb-47bf-a255-0e8101e558e6 admin admin] PATH_INFO: `/v3/projects/1345c184ec31498a80e78c6f5a709465/users/c13a8ea28fc94fb08f02a154cfc672cc/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.209685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dd2e2-8bcb-47bf-a255-0e8101e558e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1345c184ec31498a80e78c6f5a709465, user_id=c13a8ea28fc94fb08f02a154cfc672cc, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:33.213443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dd2e2-8bcb-47bf-a255-0e8101e558e6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:33.229162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1203/2410] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:43:33 2022] PUT /identity/v3/projects/1345c184ec31498a80e78c6f5a709465/users/c13a8ea28fc94fb08f02a154cfc672cc/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:43:33.233509 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48f852-5a31-467c-adce-0e199147722b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:33.244472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48f852-5a31-467c-adce-0e199147722b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:33.324395 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d17d5-7e07-4b17-808f-fcd4f4aac711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:33.325445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d17d5-7e07-4b17-808f-fcd4f4aac711 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.325795 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d17d5-7e07-4b17-808f-fcd4f4aac711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.326146 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d17d5-7e07-4b17-808f-fcd4f4aac711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.326923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d17d5-7e07-4b17-808f-fcd4f4aac711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:33.328653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d17d5-7e07-4b17-808f-fcd4f4aac711 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:33.345189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1208/2411] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:33.346018 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dd2e2-8bcb-47bf-a255-0e8101e558e6 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:33.359043 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dd2e2-8bcb-47bf-a255-0e8101e558e6 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:33.364852 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae6f7a-adec-48bf-b890-3dbf267f7791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:33.365603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae6f7a-adec-48bf-b890-3dbf267f7791 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.366077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae6f7a-adec-48bf-b890-3dbf267f7791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.366428 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae6f7a-adec-48bf-b890-3dbf267f7791 admin admin] PATH_INFO: `/v3/projects/1345c184ec31498a80e78c6f5a709465/users/c13a8ea28fc94fb08f02a154cfc672cc/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.368516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae6f7a-adec-48bf-b890-3dbf267f7791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1345c184ec31498a80e78c6f5a709465, user_id=c13a8ea28fc94fb08f02a154cfc672cc, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:33.371338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae6f7a-adec-48bf-b890-3dbf267f7791 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:33.385336 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1204/2412] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:43:33 2022] PUT /identity/v3/projects/1345c184ec31498a80e78c6f5a709465/users/c13a8ea28fc94fb08f02a154cfc672cc/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:43:33.399446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79c2e9-c2ab-4489-84e6-8c16568a78ca None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.399734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79c2e9-c2ab-4489-84e6-8c16568a78ca None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.400021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79c2e9-c2ab-4489-84e6-8c16568a78ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.434899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9f79c2e9-c2ab-4489-84e6-8c16568a78ca None None] MFA Rules not processed for user `c13a8ea28fc94fb08f02a154cfc672cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:33.436308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9f79c2e9-c2ab-4489-84e6-8c16568a78ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:33.532132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1209/2413] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:33.545124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae6f7a-adec-48bf-b890-3dbf267f7791 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:33.547009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2dae6f7a-adec-48bf-b890-3dbf267f7791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:33.617370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae6f7a-adec-48bf-b890-3dbf267f7791 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:33.621391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c695907-1e02-4ad0-aea5-bae2298bbd43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:33.622047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c695907-1e02-4ad0-aea5-bae2298bbd43 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.622421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c695907-1e02-4ad0-aea5-bae2298bbd43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.622710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c695907-1e02-4ad0-aea5-bae2298bbd43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.632949 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:33.638459 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c695907-1e02-4ad0-aea5-bae2298bbd43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:33.640549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c695907-1e02-4ad0-aea5-bae2298bbd43 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:33.654841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1205/2414] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:33.760596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79c2e9-c2ab-4489-84e6-8c16568a78ca None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:33.774443 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79c2e9-c2ab-4489-84e6-8c16568a78ca None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:33.786580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e6515-032f-4145-b14c-57a826b7c09c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:33.787189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e6515-032f-4145-b14c-57a826b7c09c service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:33.787608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e6515-032f-4145-b14c-57a826b7c09c service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:33.788084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e6515-032f-4145-b14c-57a826b7c09c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:33.824627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:33.879744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e6515-032f-4145-b14c-57a826b7c09c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:33.880730 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e6515-032f-4145-b14c-57a826b7c09c service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:33.890922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1210/2415] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:34.425830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c695907-1e02-4ad0-aea5-bae2298bbd43 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:34.427525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0c695907-1e02-4ad0-aea5-bae2298bbd43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:34.579060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c695907-1e02-4ad0-aea5-bae2298bbd43 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:34.591898 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e0d71-35f7-4fff-953e-dc9fa739fc9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:34.592444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e0d71-35f7-4fff-953e-dc9fa739fc9e service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:34.592914 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e0d71-35f7-4fff-953e-dc9fa739fc9e service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:34.593186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e0d71-35f7-4fff-953e-dc9fa739fc9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:34.595502 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-377e0d71-35f7-4fff-953e-dc9fa739fc9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:34.624352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e0d71-35f7-4fff-953e-dc9fa739fc9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:34.626254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e0d71-35f7-4fff-953e-dc9fa739fc9e service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:34.651042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1206/2416] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:43.886098 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=19 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:44.602388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e6515-032f-4145-b14c-57a826b7c09c service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:44.642457 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=37 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:44.665899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e6515-032f-4145-b14c-57a826b7c09c service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:44.670655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:44.671330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:44.671661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:44.671890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] PATH_INFO: `/v3/users/871403ff190b4b8682dd340fbd126cd0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:44.673345 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=871403ff190b4b8682dd340fbd126cd0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:44.675311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:44.731724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '871403ff190b4b8682dd340fbd126cd0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:44.739994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '871403ff190b4b8682dd340fbd126cd0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:44.751928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '871403ff190b4b8682dd340fbd126cd0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:44.752572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '871403ff190b4b8682dd340fbd126cd0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:44.753171 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '871403ff190b4b8682dd340fbd126cd0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:44.767952 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1211/2417] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:43:44 2022] DELETE /identity/v3/users/871403ff190b4b8682dd340fbd126cd0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:44.772988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331ce65-07c4-4d56-b909-004eed0844b6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:44.773281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331ce65-07c4-4d56-b909-004eed0844b6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:44.773564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331ce65-07c4-4d56-b909-004eed0844b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:44.793079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7331ce65-07c4-4d56-b909-004eed0844b6 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:44.794299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7331ce65-07c4-4d56-b909-004eed0844b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:44.854601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1207/2418] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:44.864942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:44.866412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:44.922764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717cf05-9008-4ac1-ac21-9a4d25899022 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:44.926957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92b533-0ad0-48c3-a357-3cee6bb264c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:44.927592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b533-0ad0-48c3-a357-3cee6bb264c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:44.927874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b533-0ad0-48c3-a357-3cee6bb264c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:44.928147 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b533-0ad0-48c3-a357-3cee6bb264c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:44.937471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92b533-0ad0-48c3-a357-3cee6bb264c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:44.938899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92b533-0ad0-48c3-a357-3cee6bb264c5 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:44.948473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1212/2419] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:45.270358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331ce65-07c4-4d56-b909-004eed0844b6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:45.272357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7331ce65-07c4-4d56-b909-004eed0844b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:45.284924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331ce65-07c4-4d56-b909-004eed0844b6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:45.305943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:45.306671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:45.306910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:45.307223 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] PATH_INFO: `/v3/projects/615102fbb0db49579b3982f01ee52808` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:45.308494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] RBAC: Authorizing `identity:delete_project(project_id=615102fbb0db49579b3982f01ee52808)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:45.310427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:45.360961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615102fbb0db49579b3982f01ee52808', 'request_id': 'req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:45.366796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615102fbb0db49579b3982f01ee52808', 'request_id': 'req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:45.373057 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615102fbb0db49579b3982f01ee52808', 'request_id': 'req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:45.386326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] The token cache is being invalidate because project 615102fbb0db49579b3982f01ee52808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:43:45.386616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:45.387276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:45.389003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1208/2420] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:43:45 2022] DELETE /identity/v3/projects/615102fbb0db49579b3982f01ee52808 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:43:45.393147 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92b533-0ad0-48c3-a357-3cee6bb264c5 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:45.403492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92b533-0ad0-48c3-a357-3cee6bb264c5 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:45.421159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:45.467057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca274779-b986-4e50-a776-56163d078c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:45.468328 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca274779-b986-4e50-a776-56163d078c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:45.468328 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca274779-b986-4e50-a776-56163d078c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:45.468604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca274779-b986-4e50-a776-56163d078c44 admin admin] PATH_INFO: `/v3/users/c13a8ea28fc94fb08f02a154cfc672cc` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:45.469893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca274779-b986-4e50-a776-56163d078c44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13a8ea28fc94fb08f02a154cfc672cc)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:45.473637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca274779-b986-4e50-a776-56163d078c44 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:45.529204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ca274779-b986-4e50-a776-56163d078c44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13a8ea28fc94fb08f02a154cfc672cc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:45.537102 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ca274779-b986-4e50-a776-56163d078c44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13a8ea28fc94fb08f02a154cfc672cc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:45.549629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ca274779-b986-4e50-a776-56163d078c44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13a8ea28fc94fb08f02a154cfc672cc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:45.550300 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ca274779-b986-4e50-a776-56163d078c44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13a8ea28fc94fb08f02a154cfc672cc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:45.551787 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ca274779-b986-4e50-a776-56163d078c44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13a8ea28fc94fb08f02a154cfc672cc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:45.563795 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1213/2421] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:43:45 2022] DELETE /identity/v3/users/c13a8ea28fc94fb08f02a154cfc672cc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:45.854902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:45.856395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:45.875284 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1ca8a-ac53-40cd-83fb-28c7d343537a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:45.969414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdfcab-bc33-41d8-82ba-d5447a3b7041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:45.970263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdfcab-bc33-41d8-82ba-d5447a3b7041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:45.970697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdfcab-bc33-41d8-82ba-d5447a3b7041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:45.971192 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdfcab-bc33-41d8-82ba-d5447a3b7041 admin admin] PATH_INFO: `/v3/projects/1345c184ec31498a80e78c6f5a709465` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:45.973240 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fdfcab-bc33-41d8-82ba-d5447a3b7041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1345c184ec31498a80e78c6f5a709465)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:45.982378 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fdfcab-bc33-41d8-82ba-d5447a3b7041 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:46.027593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:46.051452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33fdfcab-bc33-41d8-82ba-d5447a3b7041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1345c184ec31498a80e78c6f5a709465', 'request_id': 'req-33fdfcab-bc33-41d8-82ba-d5447a3b7041'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:46.059109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33fdfcab-bc33-41d8-82ba-d5447a3b7041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1345c184ec31498a80e78c6f5a709465', 'request_id': 'req-33fdfcab-bc33-41d8-82ba-d5447a3b7041'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:46.068566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33fdfcab-bc33-41d8-82ba-d5447a3b7041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1345c184ec31498a80e78c6f5a709465', 'request_id': 'req-33fdfcab-bc33-41d8-82ba-d5447a3b7041'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:46.099347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33fdfcab-bc33-41d8-82ba-d5447a3b7041 admin admin] The token cache is being invalidate because project 1345c184ec31498a80e78c6f5a709465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:43:46.099657 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33fdfcab-bc33-41d8-82ba-d5447a3b7041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:46.104405 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-33fdfcab-bc33-41d8-82ba-d5447a3b7041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:46.118710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1209/2422] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:43:45 2022] DELETE /identity/v3/projects/1345c184ec31498a80e78c6f5a709465 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:46.121646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91500f5-b2be-4c84-80e3-a83f40a6df51 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.122161 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91500f5-b2be-4c84-80e3-a83f40a6df51 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.122441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91500f5-b2be-4c84-80e3-a83f40a6df51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.160581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c91500f5-b2be-4c84-80e3-a83f40a6df51 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:46.162591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c91500f5-b2be-4c84-80e3-a83f40a6df51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:46.257834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1214/2423] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:46.265832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e853e-28ef-457f-906e-e1af52902438 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.266144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e853e-28ef-457f-906e-e1af52902438 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.266423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e853e-28ef-457f-906e-e1af52902438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.292903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-105e853e-28ef-457f-906e-e1af52902438 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:46.294560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-105e853e-28ef-457f-906e-e1af52902438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:46.304124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1210/2424] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:43:46.308320 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91500f5-b2be-4c84-80e3-a83f40a6df51 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:46.318648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91500f5-b2be-4c84-80e3-a83f40a6df51 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:46.323987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1879107-ae1a-408c-864a-326127c61698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:46.324633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1879107-ae1a-408c-864a-326127c61698 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.324916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1879107-ae1a-408c-864a-326127c61698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.325190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1879107-ae1a-408c-864a-326127c61698 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.325883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1879107-ae1a-408c-864a-326127c61698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:46.326276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1879107-ae1a-408c-864a-326127c61698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:46.327927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1879107-ae1a-408c-864a-326127c61698 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:46.341920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1215/2425] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:43:46.346449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e853e-28ef-457f-906e-e1af52902438 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:46.355643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e853e-28ef-457f-906e-e1af52902438 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:46.360935 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246f8e2-58d3-44b1-9bc3-324e98d7334b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:46.361445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246f8e2-58d3-44b1-9bc3-324e98d7334b admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.361808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246f8e2-58d3-44b1-9bc3-324e98d7334b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.362058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246f8e2-58d3-44b1-9bc3-324e98d7334b admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.362753 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246f8e2-58d3-44b1-9bc3-324e98d7334b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:46.365515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246f8e2-58d3-44b1-9bc3-324e98d7334b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:46.410138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1211/2426] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:43:46 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:46.411565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1879107-ae1a-408c-864a-326127c61698 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:46.422882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1879107-ae1a-408c-864a-326127c61698 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:46.455281 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:46.493814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcf84f-5748-41bb-bf1e-bd27cca185bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:46.494356 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcf84f-5748-41bb-bf1e-bd27cca185bc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.494640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcf84f-5748-41bb-bf1e-bd27cca185bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.494917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcf84f-5748-41bb-bf1e-bd27cca185bc admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.495626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcf84f-5748-41bb-bf1e-bd27cca185bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:46.497624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcf84f-5748-41bb-bf1e-bd27cca185bc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:46.521681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1216/2427] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:43:46 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:46.526072 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246f8e2-58d3-44b1-9bc3-324e98d7334b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:46.540117 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246f8e2-58d3-44b1-9bc3-324e98d7334b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:46.547350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902f7b0-863d-4020-b82e-60d26006dfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:46.548055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902f7b0-863d-4020-b82e-60d26006dfac admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.548221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902f7b0-863d-4020-b82e-60d26006dfac admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.548484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902f7b0-863d-4020-b82e-60d26006dfac admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.549216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902f7b0-863d-4020-b82e-60d26006dfac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:46.551660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902f7b0-863d-4020-b82e-60d26006dfac admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:46.568423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1212/2428] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:46.570029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcf84f-5748-41bb-bf1e-bd27cca185bc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:46.580820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcf84f-5748-41bb-bf1e-bd27cca185bc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:46.593616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8f296-c269-4fa2-823d-cf00e6f89687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:46.594284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8f296-c269-4fa2-823d-cf00e6f89687 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.595416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8f296-c269-4fa2-823d-cf00e6f89687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.595709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8f296-c269-4fa2-823d-cf00e6f89687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.596417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8f296-c269-4fa2-823d-cf00e6f89687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:46.600507 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8f296-c269-4fa2-823d-cf00e6f89687 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:46.616627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1217/2429] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:46.618489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902f7b0-863d-4020-b82e-60d26006dfac admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:46.631383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902f7b0-863d-4020-b82e-60d26006dfac admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:46.636837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c836e3-db65-4420-96ae-fe3d96f905ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:46.637520 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c836e3-db65-4420-96ae-fe3d96f905ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.637807 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c836e3-db65-4420-96ae-fe3d96f905ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.638195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c836e3-db65-4420-96ae-fe3d96f905ed admin admin] PATH_INFO: `/v3/projects/61a3486fc2924ea3b4b617f54ea64982/users/870416b02b4d4c75b7ff3640c800bb38/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.659993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c836e3-db65-4420-96ae-fe3d96f905ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a3486fc2924ea3b4b617f54ea64982, user_id=870416b02b4d4c75b7ff3640c800bb38, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:46.663636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c836e3-db65-4420-96ae-fe3d96f905ed admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:46.685412 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1213/2430] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:43:46 2022] PUT /identity/v3/projects/61a3486fc2924ea3b4b617f54ea64982/users/870416b02b4d4c75b7ff3640c800bb38/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:46.698487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a1991-e834-4c9f-9e72-79402367f9ae None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.698788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a1991-e834-4c9f-9e72-79402367f9ae None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.699163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a1991-e834-4c9f-9e72-79402367f9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.742354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cc6a1991-e834-4c9f-9e72-79402367f9ae None None] MFA Rules not processed for user `870416b02b4d4c75b7ff3640c800bb38`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:46.743696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cc6a1991-e834-4c9f-9e72-79402367f9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:46.837500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1218/2431] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:46.843275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c836e3-db65-4420-96ae-fe3d96f905ed admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:46.908984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c836e3-db65-4420-96ae-fe3d96f905ed admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:46.919952 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa3edf-8236-4c55-9257-8f81af4a8e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:46.921282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa3edf-8236-4c55-9257-8f81af4a8e26 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:46.921569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa3edf-8236-4c55-9257-8f81af4a8e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:46.921851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa3edf-8236-4c55-9257-8f81af4a8e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:46.922528 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa3edf-8236-4c55-9257-8f81af4a8e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:46.924608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa3edf-8236-4c55-9257-8f81af4a8e26 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:46.955363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1214/2432] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:43:46 2022] POST /identity/v3/projects => generated 369 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:46.968424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a1991-e834-4c9f-9e72-79402367f9ae None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:46.987542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a1991-e834-4c9f-9e72-79402367f9ae None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:47.050050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f5db7-930c-49a9-8c68-4c0565fd14b6 None None] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:47.050347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f5db7-930c-49a9-8c68-4c0565fd14b6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:47.050698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f5db7-930c-49a9-8c68-4c0565fd14b6 None None] PATH_INFO: `/` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:47.053523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1215/2433] 10.222.0.40 () {54 vars in 921 bytes} [Mon Aug 1 07:43:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 07:43:47.061694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df76d71-7494-4048-befa-6f120e5dcc96 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:47.062029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df76d71-7494-4048-befa-6f120e5dcc96 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:47.062317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df76d71-7494-4048-befa-6f120e5dcc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:47.090747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6df76d71-7494-4048-befa-6f120e5dcc96 None None] MFA Rules not processed for user `abdcd6d09fee45c696449641ea36eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:47.092412 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6df76d71-7494-4048-befa-6f120e5dcc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:47.115041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e4181-1bb5-471f-a9e7-2f0d3dc2cce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:47.117337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e4181-1bb5-471f-a9e7-2f0d3dc2cce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:47.118366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e4181-1bb5-471f-a9e7-2f0d3dc2cce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:47.118366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e4181-1bb5-471f-a9e7-2f0d3dc2cce0 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:47.120572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e4181-1bb5-471f-a9e7-2f0d3dc2cce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:47.142452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e4181-1bb5-471f-a9e7-2f0d3dc2cce0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:47.151398 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:47.198035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1219/2434] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:43:46 2022] POST /identity/v3/users => generated 291 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:47.201369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e4181-1bb5-471f-a9e7-2f0d3dc2cce0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:47.234674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e4181-1bb5-471f-a9e7-2f0d3dc2cce0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:47.243107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988c8b8-ba00-406f-97a3-3ac49e2410aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:47.243891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988c8b8-ba00-406f-97a3-3ac49e2410aa admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:47.244279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988c8b8-ba00-406f-97a3-3ac49e2410aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:47.244655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988c8b8-ba00-406f-97a3-3ac49e2410aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:47.245751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b988c8b8-ba00-406f-97a3-3ac49e2410aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:47.247640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b988c8b8-ba00-406f-97a3-3ac49e2410aa admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:47.291094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1220/2435] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:47.291302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1216/2436] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Aug 1 07:43:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:43:47.291415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988c8b8-ba00-406f-97a3-3ac49e2410aa admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:47.300007 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df76d71-7494-4048-befa-6f120e5dcc96 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:47.302668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6df76d71-7494-4048-befa-6f120e5dcc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:47.310401 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988c8b8-ba00-406f-97a3-3ac49e2410aa admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:47.324986 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cb3cf-0627-48aa-85bf-8ba2c6f15727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:47.325818 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cb3cf-0627-48aa-85bf-8ba2c6f15727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:47.326242 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cb3cf-0627-48aa-85bf-8ba2c6f15727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:47.326592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cb3cf-0627-48aa-85bf-8ba2c6f15727 admin admin] PATH_INFO: `/v3/projects/b8c0ca643d6c40e6a3815a97d639bf1d/users/f1ae4b536a164b27ac1e738768d29593/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:47.354833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:47.356791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cb3cf-0627-48aa-85bf-8ba2c6f15727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c0ca643d6c40e6a3815a97d639bf1d, user_id=f1ae4b536a164b27ac1e738768d29593, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:47.371911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cb3cf-0627-48aa-85bf-8ba2c6f15727 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:47.395113 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1221/2437] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:43:47 2022] PUT /identity/v3/projects/b8c0ca643d6c40e6a3815a97d639bf1d/users/f1ae4b536a164b27ac1e738768d29593/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:47.398532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cb3cf-0627-48aa-85bf-8ba2c6f15727 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:47.437138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df76d71-7494-4048-befa-6f120e5dcc96 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:47.486024 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cb3cf-0627-48aa-85bf-8ba2c6f15727 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:47.491917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e2332-8043-4982-83db-6e68bf2e04df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:47.492638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e2332-8043-4982-83db-6e68bf2e04df admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:47.492888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e2332-8043-4982-83db-6e68bf2e04df admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:47.493174 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e2332-8043-4982-83db-6e68bf2e04df admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:47.493873 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e2332-8043-4982-83db-6e68bf2e04df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:47.495553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e2332-8043-4982-83db-6e68bf2e04df admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:47.498247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcefb3-3676-4c2e-bd13-f39ff9fc0e38 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f57fa108c3d478a9976f4aea5f33bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:47.498821 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcefb3-3676-4c2e-bd13-f39ff9fc0e38 service glance] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:47.499172 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcefb3-3676-4c2e-bd13-f39ff9fc0e38 service glance] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:47.499452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcefb3-3676-4c2e-bd13-f39ff9fc0e38 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:47.505984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1222/2438] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:47.510566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e2332-8043-4982-83db-6e68bf2e04df admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:47.521935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e2332-8043-4982-83db-6e68bf2e04df admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:47.526827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea4c30-8767-4928-9a86-acaee8806b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:47.527625 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea4c30-8767-4928-9a86-acaee8806b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:47.527905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea4c30-8767-4928-9a86-acaee8806b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:47.528231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea4c30-8767-4928-9a86-acaee8806b19 admin admin] PATH_INFO: `/v3/projects/b8c0ca643d6c40e6a3815a97d639bf1d/users/f1ae4b536a164b27ac1e738768d29593/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:47.530407 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea4c30-8767-4928-9a86-acaee8806b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c0ca643d6c40e6a3815a97d639bf1d, user_id=f1ae4b536a164b27ac1e738768d29593, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:47.533116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea4c30-8767-4928-9a86-acaee8806b19 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:47.546724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1223/2439] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:43:47 2022] PUT /identity/v3/projects/b8c0ca643d6c40e6a3815a97d639bf1d/users/f1ae4b536a164b27ac1e738768d29593/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:47.560001 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fae4c7-6a21-4c2b-ac83-017d215a0e5a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:47.560299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fae4c7-6a21-4c2b-ac83-017d215a0e5a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:47.560579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fae4c7-6a21-4c2b-ac83-017d215a0e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:47.578068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcefb3-3676-4c2e-bd13-f39ff9fc0e38 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:47.579568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcefb3-3676-4c2e-bd13-f39ff9fc0e38 service glance] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:47.591265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c3fae4c7-6a21-4c2b-ac83-017d215a0e5a None None] MFA Rules not processed for user `f1ae4b536a164b27ac1e738768d29593`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:47.591432 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1217/2440] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:47.592559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c3fae4c7-6a21-4c2b-ac83-017d215a0e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:47.680628 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1224/2441] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:47 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:43:47.690639 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcefb3-3676-4c2e-bd13-f39ff9fc0e38 service glance] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:47.692871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3afcefb3-3676-4c2e-bd13-f39ff9fc0e38 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:47.751249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:47.802937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcefb3-3676-4c2e-bd13-f39ff9fc0e38 service glance] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:47.853805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b8df1-0c62-4bfb-b692-220bef1a0692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:47.854530 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b8df1-0c62-4bfb-b692-220bef1a0692 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:47.854802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b8df1-0c62-4bfb-b692-220bef1a0692 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:47.855105 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b8df1-0c62-4bfb-b692-220bef1a0692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:47.865000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b8df1-0c62-4bfb-b692-220bef1a0692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:47.866424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b8df1-0c62-4bfb-b692-220bef1a0692 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:47.876100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1218/2442] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:47 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:47.920466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fae4c7-6a21-4c2b-ac83-017d215a0e5a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:47.921942 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c3fae4c7-6a21-4c2b-ac83-017d215a0e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:47.934599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fae4c7-6a21-4c2b-ac83-017d215a0e5a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:47.938223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d2cd8-fb36-4678-b37f-27913f05f664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:47.938829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d2cd8-fb36-4678-b37f-27913f05f664 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:47.939251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d2cd8-fb36-4678-b37f-27913f05f664 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:47.939538 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d2cd8-fb36-4678-b37f-27913f05f664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:48.018588 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d2cd8-fb36-4678-b37f-27913f05f664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:48.020232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d2cd8-fb36-4678-b37f-27913f05f664 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:48.023932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:48.039482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1225/2443] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:48.956248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac6bd1-c787-4782-8378-abd72edfc321 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:48.956904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac6bd1-c787-4782-8378-abd72edfc321 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:48.957069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac6bd1-c787-4782-8378-abd72edfc321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:48.979853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-aaac6bd1-c787-4782-8378-abd72edfc321 None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:48.981474 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-aaac6bd1-c787-4782-8378-abd72edfc321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:49.062101 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1219/2444] 10.222.0.40 () {58 vars in 967 bytes} [Mon Aug 1 07:43:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:49.076663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d2cd8-fb36-4678-b37f-27913f05f664 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:49.078922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-648d2cd8-fb36-4678-b37f-27913f05f664 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:49.157032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d2cd8-fb36-4678-b37f-27913f05f664 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:49.164495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc9ee0-2b89-49c5-833e-7c83b90eaadb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:49.165224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9ee0-2b89-49c5-833e-7c83b90eaadb service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:49.165605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9ee0-2b89-49c5-833e-7c83b90eaadb service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:49.166046 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9ee0-2b89-49c5-833e-7c83b90eaadb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:49.176354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc9ee0-2b89-49c5-833e-7c83b90eaadb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:49.178039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc9ee0-2b89-49c5-833e-7c83b90eaadb service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:49.187550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1226/2445] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:43:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:43:49.545568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f04b01-e5b6-419e-aa8d-5f00a10c1f3a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:49.546480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f04b01-e5b6-419e-aa8d-5f00a10c1f3a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:49.547201 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f04b01-e5b6-419e-aa8d-5f00a10c1f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:49.574474 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-52f04b01-e5b6-419e-aa8d-5f00a10c1f3a None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:49.576118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-52f04b01-e5b6-419e-aa8d-5f00a10c1f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:49.585287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1220/2446] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:43:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:49.602024 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc9ee0-2b89-49c5-833e-7c83b90eaadb service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:49.619737 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc9ee0-2b89-49c5-833e-7c83b90eaadb service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:49.633009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a5730-7f23-48af-b143-9c45cb36caa0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:49.633756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a5730-7f23-48af-b143-9c45cb36caa0 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:49.634216 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a5730-7f23-48af-b143-9c45cb36caa0 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:49.634481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a5730-7f23-48af-b143-9c45cb36caa0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:49.662058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a5730-7f23-48af-b143-9c45cb36caa0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:49.663757 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a5730-7f23-48af-b143-9c45cb36caa0 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:49.676352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1227/2447] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:43:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 07:43:52.479311 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f04b01-e5b6-419e-aa8d-5f00a10c1f3a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:52.602555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f04b01-e5b6-419e-aa8d-5f00a10c1f3a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:52.608780 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:52.609573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:52.609973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:52.618458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] PATH_INFO: `/v3/users/870416b02b4d4c75b7ff3640c800bb38` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:52.620257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=870416b02b4d4c75b7ff3640c800bb38)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:52.622570 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:52.675856 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:52.721474 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870416b02b4d4c75b7ff3640c800bb38'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:52.754212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870416b02b4d4c75b7ff3640c800bb38'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:52.754997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870416b02b4d4c75b7ff3640c800bb38'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:52.772243 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870416b02b4d4c75b7ff3640c800bb38'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:52.784291 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870416b02b4d4c75b7ff3640c800bb38'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:52.796060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1221/2448] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:43:52 2022] DELETE /identity/v3/users/870416b02b4d4c75b7ff3640c800bb38 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:52.801171 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f499588-be30-4355-a339-b04c6f585762 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:52.801989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f499588-be30-4355-a339-b04c6f585762 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:52.801989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f499588-be30-4355-a339-b04c6f585762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:52.848014 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9f499588-be30-4355-a339-b04c6f585762 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:52.849409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9f499588-be30-4355-a339-b04c6f585762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:53.052748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1228/2449] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:53.078421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:53.079947 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:53.205595 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb46d2-7133-4d92-8e22-0732c1d58395 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:53.223831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7aac8-4553-4447-b487-d337797be20e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:53.224486 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7aac8-4553-4447-b487-d337797be20e service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:53.224774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7aac8-4553-4447-b487-d337797be20e service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:53.225060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7aac8-4553-4447-b487-d337797be20e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:53.259655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7aac8-4553-4447-b487-d337797be20e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:53.261137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7aac8-4553-4447-b487-d337797be20e service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:53.282265 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1222/2450] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:54.045707 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f499588-be30-4355-a339-b04c6f585762 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:54.047284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9f499588-be30-4355-a339-b04c6f585762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:54.079824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f499588-be30-4355-a339-b04c6f585762 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:54.094967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:54.100143 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:54.100442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:54.100748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] PATH_INFO: `/v3/projects/61a3486fc2924ea3b4b617f54ea64982` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:54.104593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a3486fc2924ea3b4b617f54ea64982)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:54.106539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:54.236716 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a3486fc2924ea3b4b617f54ea64982', 'request_id': 'req-5ec25c07-36d8-411c-affa-d753062f5563'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:54.255675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a3486fc2924ea3b4b617f54ea64982', 'request_id': 'req-5ec25c07-36d8-411c-affa-d753062f5563'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:54.272777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a3486fc2924ea3b4b617f54ea64982', 'request_id': 'req-5ec25c07-36d8-411c-affa-d753062f5563'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:54.290855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:54.306475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] The token cache is being invalidate because project 61a3486fc2924ea3b4b617f54ea64982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:43:54.308365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:54.310215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:54.315189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1229/2451] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:43:54 2022] DELETE /identity/v3/projects/61a3486fc2924ea3b4b617f54ea64982 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:54.316364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7aac8-4553-4447-b487-d337797be20e service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:54.334787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7aac8-4553-4447-b487-d337797be20e service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:54.496257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ffef4-302d-4dce-a771-eb919d7b119d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:54.497060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ffef4-302d-4dce-a771-eb919d7b119d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:54.497349 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ffef4-302d-4dce-a771-eb919d7b119d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:54.497626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ffef4-302d-4dce-a771-eb919d7b119d admin admin] PATH_INFO: `/v3/users/f1ae4b536a164b27ac1e738768d29593` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:54.499459 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ffef4-302d-4dce-a771-eb919d7b119d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1ae4b536a164b27ac1e738768d29593)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:54.501873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ffef4-302d-4dce-a771-eb919d7b119d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:54.604889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:54.612276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2b2ffef4-302d-4dce-a771-eb919d7b119d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1ae4b536a164b27ac1e738768d29593'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:54.650068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2b2ffef4-302d-4dce-a771-eb919d7b119d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1ae4b536a164b27ac1e738768d29593'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:54.650916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2b2ffef4-302d-4dce-a771-eb919d7b119d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1ae4b536a164b27ac1e738768d29593'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:54.668907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2b2ffef4-302d-4dce-a771-eb919d7b119d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1ae4b536a164b27ac1e738768d29593'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:54.691016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-2b2ffef4-302d-4dce-a771-eb919d7b119d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1ae4b536a164b27ac1e738768d29593'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:54.705235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1223/2452] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:43:54 2022] DELETE /identity/v3/users/f1ae4b536a164b27ac1e738768d29593 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:55.101326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:55.102914 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:55.123024 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec25c07-36d8-411c-affa-d753062f5563 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:55.250675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c99cb-be9e-41e0-9a61-5c941b496443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:55.251423 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c99cb-be9e-41e0-9a61-5c941b496443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:55.251766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c99cb-be9e-41e0-9a61-5c941b496443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:55.252051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c99cb-be9e-41e0-9a61-5c941b496443 admin admin] PATH_INFO: `/v3/projects/b8c0ca643d6c40e6a3815a97d639bf1d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:55.254459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c99cb-be9e-41e0-9a61-5c941b496443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8c0ca643d6c40e6a3815a97d639bf1d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:55.257265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c99cb-be9e-41e0-9a61-5c941b496443 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:55.357736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-699c99cb-be9e-41e0-9a61-5c941b496443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8c0ca643d6c40e6a3815a97d639bf1d', 'request_id': 'req-699c99cb-be9e-41e0-9a61-5c941b496443'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:55.370045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-699c99cb-be9e-41e0-9a61-5c941b496443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8c0ca643d6c40e6a3815a97d639bf1d', 'request_id': 'req-699c99cb-be9e-41e0-9a61-5c941b496443'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:55.381948 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-699c99cb-be9e-41e0-9a61-5c941b496443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8c0ca643d6c40e6a3815a97d639bf1d', 'request_id': 'req-699c99cb-be9e-41e0-9a61-5c941b496443'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:55.406532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-699c99cb-be9e-41e0-9a61-5c941b496443 admin admin] The token cache is being invalidate because project b8c0ca643d6c40e6a3815a97d639bf1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:43:55.407332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-699c99cb-be9e-41e0-9a61-5c941b496443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:55.408521 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-699c99cb-be9e-41e0-9a61-5c941b496443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:55.410452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1230/2453] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:43:55 2022] DELETE /identity/v3/projects/b8c0ca643d6c40e6a3815a97d639bf1d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:55.415600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586c59e-e9b9-46f2-b304-09a539b5deb6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:55.416064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586c59e-e9b9-46f2-b304-09a539b5deb6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:55.416314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586c59e-e9b9-46f2-b304-09a539b5deb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:55.450814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7586c59e-e9b9-46f2-b304-09a539b5deb6 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:55.452154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7586c59e-e9b9-46f2-b304-09a539b5deb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:55.537658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1224/2454] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:43:55.541312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c909a-5f99-4f4f-8d2a-be0a08615bf1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:55.541936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c909a-5f99-4f4f-8d2a-be0a08615bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:55.542420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c909a-5f99-4f4f-8d2a-be0a08615bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:55.590244 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-656c909a-5f99-4f4f-8d2a-be0a08615bf1 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:55.592535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-656c909a-5f99-4f4f-8d2a-be0a08615bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:55.612128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1231/2455] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:55.618493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586c59e-e9b9-46f2-b304-09a539b5deb6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:55.646524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586c59e-e9b9-46f2-b304-09a539b5deb6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:55.657927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de021f2-00d2-4c4e-a54d-7aba365ae801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:55.658544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de021f2-00d2-4c4e-a54d-7aba365ae801 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:55.658977 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de021f2-00d2-4c4e-a54d-7aba365ae801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:55.659283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de021f2-00d2-4c4e-a54d-7aba365ae801 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:55.661285 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de021f2-00d2-4c4e-a54d-7aba365ae801 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:43:55.661603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de021f2-00d2-4c4e-a54d-7aba365ae801 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:55.668365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de021f2-00d2-4c4e-a54d-7aba365ae801 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:55.719150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1225/2456] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:43:55.723211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c909a-5f99-4f4f-8d2a-be0a08615bf1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:55.741085 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c909a-5f99-4f4f-8d2a-be0a08615bf1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:55.760355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6cb64-a1e6-4302-869f-7a881f0f0895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:55.761337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6cb64-a1e6-4302-869f-7a881f0f0895 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:55.761617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6cb64-a1e6-4302-869f-7a881f0f0895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:55.764218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6cb64-a1e6-4302-869f-7a881f0f0895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:55.765522 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6cb64-a1e6-4302-869f-7a881f0f0895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:55.773720 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6cb64-a1e6-4302-869f-7a881f0f0895 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:55.852766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1232/2457] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:43:55 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:55.856671 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de021f2-00d2-4c4e-a54d-7aba365ae801 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:55.878437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de021f2-00d2-4c4e-a54d-7aba365ae801 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:56.051555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a225096-2003-44a3-8a41-0d245b5dec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:56.052054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a225096-2003-44a3-8a41-0d245b5dec69 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:56.052409 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a225096-2003-44a3-8a41-0d245b5dec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:56.052612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a225096-2003-44a3-8a41-0d245b5dec69 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:56.053294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a225096-2003-44a3-8a41-0d245b5dec69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:56.055275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a225096-2003-44a3-8a41-0d245b5dec69 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:56.134278 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1226/2458] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:43:55 2022] POST /identity/v3/users => generated 293 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:43:56.139295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6cb64-a1e6-4302-869f-7a881f0f0895 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:56.155704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6cb64-a1e6-4302-869f-7a881f0f0895 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:56.194434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f07fb6-0af6-4d9e-88cb-5a2a78b2a508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:56.195404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f07fb6-0af6-4d9e-88cb-5a2a78b2a508 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:56.197154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f07fb6-0af6-4d9e-88cb-5a2a78b2a508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:56.198155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f07fb6-0af6-4d9e-88cb-5a2a78b2a508 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:56.203779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f07fb6-0af6-4d9e-88cb-5a2a78b2a508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:56.208445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f07fb6-0af6-4d9e-88cb-5a2a78b2a508 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:56.263436 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1233/2459] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:56.263668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a225096-2003-44a3-8a41-0d245b5dec69 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:56.285888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a225096-2003-44a3-8a41-0d245b5dec69 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:56.304147 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5223844-a2b7-41b2-a74e-405d8a72b717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:56.305144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5223844-a2b7-41b2-a74e-405d8a72b717 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:56.305551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5223844-a2b7-41b2-a74e-405d8a72b717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:56.305952 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5223844-a2b7-41b2-a74e-405d8a72b717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:56.306807 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5223844-a2b7-41b2-a74e-405d8a72b717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:56.309415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5223844-a2b7-41b2-a74e-405d8a72b717 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:56.338609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1227/2460] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:56.340639 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f07fb6-0af6-4d9e-88cb-5a2a78b2a508 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:56.371597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f07fb6-0af6-4d9e-88cb-5a2a78b2a508 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:56.384881 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f434aca-46a4-46f8-8867-302346952813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:56.385571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f434aca-46a4-46f8-8867-302346952813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:56.385859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f434aca-46a4-46f8-8867-302346952813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:56.386140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f434aca-46a4-46f8-8867-302346952813 admin admin] PATH_INFO: `/v3/projects/c617f4a84fcc445a90a51e499e908f73/users/8b7ff93ddb3b48048b8487d67580e17b/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:56.435872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f434aca-46a4-46f8-8867-302346952813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c617f4a84fcc445a90a51e499e908f73, user_id=8b7ff93ddb3b48048b8487d67580e17b, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:56.438594 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f434aca-46a4-46f8-8867-302346952813 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:56.465867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1234/2461] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:43:56 2022] PUT /identity/v3/projects/c617f4a84fcc445a90a51e499e908f73/users/8b7ff93ddb3b48048b8487d67580e17b/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:56.478451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257330e-16da-4842-8a5f-b53a573c8f88 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:56.479747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257330e-16da-4842-8a5f-b53a573c8f88 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:56.480212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257330e-16da-4842-8a5f-b53a573c8f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:56.530827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b257330e-16da-4842-8a5f-b53a573c8f88 None None] MFA Rules not processed for user `8b7ff93ddb3b48048b8487d67580e17b`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:56.532412 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b257330e-16da-4842-8a5f-b53a573c8f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:56.553715 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:56.663206 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1228/2462] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:56.667993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f434aca-46a4-46f8-8867-302346952813 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:56.697879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:56.741120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f434aca-46a4-46f8-8867-302346952813 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:56.746730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257330e-16da-4842-8a5f-b53a573c8f88 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:56.747027 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022f8fd-eb66-4348-80d6-c94dd639c6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:56.747593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022f8fd-eb66-4348-80d6-c94dd639c6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:56.748054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022f8fd-eb66-4348-80d6-c94dd639c6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:56.748163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022f8fd-eb66-4348-80d6-c94dd639c6b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:56.748900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022f8fd-eb66-4348-80d6-c94dd639c6b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:56.750685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b257330e-16da-4842-8a5f-b53a573c8f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:56.751282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022f8fd-eb66-4348-80d6-c94dd639c6b7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:56.791552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1235/2463] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:43:56 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:56.796592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022f8fd-eb66-4348-80d6-c94dd639c6b7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:56.869880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022f8fd-eb66-4348-80d6-c94dd639c6b7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:56.873325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257330e-16da-4842-8a5f-b53a573c8f88 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:56.877680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc346-daf7-47d2-ba5d-a2b0806f94ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:56.879796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc346-daf7-47d2-ba5d-a2b0806f94ef admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:56.880096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc346-daf7-47d2-ba5d-a2b0806f94ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:56.880378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc346-daf7-47d2-ba5d-a2b0806f94ef admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:56.881088 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bc346-daf7-47d2-ba5d-a2b0806f94ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:56.883041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bc346-daf7-47d2-ba5d-a2b0806f94ef admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:56.907456 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1236/2464] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:43:56 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:43:56.911403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc346-daf7-47d2-ba5d-a2b0806f94ef admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:56.923705 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc346-daf7-47d2-ba5d-a2b0806f94ef admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:56.931086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4c4fd-ee65-4a48-a863-4544fff475a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:56.931907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4c4fd-ee65-4a48-a863-4544fff475a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:56.932335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4c4fd-ee65-4a48-a863-4544fff475a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:56.932716 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4c4fd-ee65-4a48-a863-4544fff475a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:56.933575 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4c4fd-ee65-4a48-a863-4544fff475a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:56.935331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4c4fd-ee65-4a48-a863-4544fff475a8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:56.952070 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1237/2465] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:43:56.954921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4c4fd-ee65-4a48-a863-4544fff475a8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:56.955555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef69375-3bbf-4d9d-9598-a347a16c00b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928c5d1284b74f1291c031323bb03a20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:56.956199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef69375-3bbf-4d9d-9598-a347a16c00b4 service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:56.956554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef69375-3bbf-4d9d-9598-a347a16c00b4 service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:56.956817 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef69375-3bbf-4d9d-9598-a347a16c00b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:56.958781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6ef69375-3bbf-4d9d-9598-a347a16c00b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:56.970829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4c4fd-ee65-4a48-a863-4544fff475a8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:56.992031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b2e1e-7d65-4cdf-b912-e83a8b927121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:56.992725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef69375-3bbf-4d9d-9598-a347a16c00b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:56.992866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b2e1e-7d65-4cdf-b912-e83a8b927121 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:56.993081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b2e1e-7d65-4cdf-b912-e83a8b927121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:56.993363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b2e1e-7d65-4cdf-b912-e83a8b927121 admin admin] PATH_INFO: `/v3/projects/9acf1b27daf2479c9c8ef0d2543e9369/users/ba48b720e954465c8d1838ba9c320dce/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:56.995773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef69375-3bbf-4d9d-9598-a347a16c00b4 service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:57.023971 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1229/2466] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:43:57.045416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b2e1e-7d65-4cdf-b912-e83a8b927121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9acf1b27daf2479c9c8ef0d2543e9369, user_id=ba48b720e954465c8d1838ba9c320dce, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:57.053371 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b2e1e-7d65-4cdf-b912-e83a8b927121 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:57.080206 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1238/2467] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:43:56 2022] PUT /identity/v3/projects/9acf1b27daf2479c9c8ef0d2543e9369/users/ba48b720e954465c8d1838ba9c320dce/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:57.081485 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef69375-3bbf-4d9d-9598-a347a16c00b4 service placement] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:57.095100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef69375-3bbf-4d9d-9598-a347a16c00b4 service placement] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:57.179159 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:57.193740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16cdb5-de58-4ddf-a7c9-9541d198a295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:57.194631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16cdb5-de58-4ddf-a7c9-9541d198a295 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:57.195015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16cdb5-de58-4ddf-a7c9-9541d198a295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:57.195422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16cdb5-de58-4ddf-a7c9-9541d198a295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:57.196294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16cdb5-de58-4ddf-a7c9-9541d198a295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:57.198603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16cdb5-de58-4ddf-a7c9-9541d198a295 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:57.219129 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1230/2468] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:43:57.220940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b2e1e-7d65-4cdf-b912-e83a8b927121 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:57.238651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b2e1e-7d65-4cdf-b912-e83a8b927121 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:57.245906 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e4f25-8cdb-4017-8c3d-243c0643c929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:57.246717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e4f25-8cdb-4017-8c3d-243c0643c929 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:57.247307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e4f25-8cdb-4017-8c3d-243c0643c929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:57.247733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e4f25-8cdb-4017-8c3d-243c0643c929 admin admin] PATH_INFO: `/v3/projects/9acf1b27daf2479c9c8ef0d2543e9369/users/ba48b720e954465c8d1838ba9c320dce/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:57.254445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e4f25-8cdb-4017-8c3d-243c0643c929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9acf1b27daf2479c9c8ef0d2543e9369, user_id=ba48b720e954465c8d1838ba9c320dce, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:57.257551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e4f25-8cdb-4017-8c3d-243c0643c929 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:57.279119 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1239/2469] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:43:57 2022] PUT /identity/v3/projects/9acf1b27daf2479c9c8ef0d2543e9369/users/ba48b720e954465c8d1838ba9c320dce/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:57.285365 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4448dde-3f52-4d5b-9ee5-67859142225a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:57.285917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4448dde-3f52-4d5b-9ee5-67859142225a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:57.286377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4448dde-3f52-4d5b-9ee5-67859142225a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:57.342560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b4448dde-3f52-4d5b-9ee5-67859142225a None None] MFA Rules not processed for user `ba48b720e954465c8d1838ba9c320dce`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:57.344471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b4448dde-3f52-4d5b-9ee5-67859142225a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:57.508299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1231/2470] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:57.521536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e4f25-8cdb-4017-8c3d-243c0643c929 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:57.523684 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-112e4f25-8cdb-4017-8c3d-243c0643c929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:57.660449 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e4f25-8cdb-4017-8c3d-243c0643c929 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:57.665434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159ee1f-e692-40f5-be89-23edba3d3190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:57.667033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ee1f-e692-40f5-be89-23edba3d3190 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:57.667624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ee1f-e692-40f5-be89-23edba3d3190 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:57.668152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ee1f-e692-40f5-be89-23edba3d3190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:57.675834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:57.681366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159ee1f-e692-40f5-be89-23edba3d3190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:57.685128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159ee1f-e692-40f5-be89-23edba3d3190 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:57.697910 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1240/2471] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:43:57.744559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4448dde-3f52-4d5b-9ee5-67859142225a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:57.892402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4448dde-3f52-4d5b-9ee5-67859142225a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:57.897411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:57.898027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:57.898313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:57.898636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] PATH_INFO: `/v3/users/8b7ff93ddb3b48048b8487d67580e17b` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:57.899792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7ff93ddb3b48048b8487d67580e17b)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:57.902622 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:58.019846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7ff93ddb3b48048b8487d67580e17b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:58.042124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7ff93ddb3b48048b8487d67580e17b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:58.044988 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7ff93ddb3b48048b8487d67580e17b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:58.060920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7ff93ddb3b48048b8487d67580e17b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:58.062718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:58.074055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7ff93ddb3b48048b8487d67580e17b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:58.093713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1232/2472] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:43:57 2022] DELETE /identity/v3/users/8b7ff93ddb3b48048b8487d67580e17b => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:58.099473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7900a96-e147-4d9e-af7a-ab14973021e8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:58.100404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7900a96-e147-4d9e-af7a-ab14973021e8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:58.100699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7900a96-e147-4d9e-af7a-ab14973021e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:58.122442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f7900a96-e147-4d9e-af7a-ab14973021e8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:58.124091 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f7900a96-e147-4d9e-af7a-ab14973021e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:58.208551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1241/2473] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:43:58.218354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:58.219827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:58.318269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6315b1-bc6c-4184-af55-d52931a208f9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:58.322597 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5b772-2098-46af-b0c4-12ed07a33539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:58.323269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5b772-2098-46af-b0c4-12ed07a33539 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:58.323554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5b772-2098-46af-b0c4-12ed07a33539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:58.323827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5b772-2098-46af-b0c4-12ed07a33539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:58.337714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5b772-2098-46af-b0c4-12ed07a33539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:58.339335 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5b772-2098-46af-b0c4-12ed07a33539 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:58.355399 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1233/2474] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:43:58.928838 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7900a96-e147-4d9e-af7a-ab14973021e8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:58.930276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f7900a96-e147-4d9e-af7a-ab14973021e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:58.945682 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7900a96-e147-4d9e-af7a-ab14973021e8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:58.952140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:58.954438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:58.954989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:58.955598 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] PATH_INFO: `/v3/projects/c617f4a84fcc445a90a51e499e908f73` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:58.958205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c617f4a84fcc445a90a51e499e908f73)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:58.961367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:59.031600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c617f4a84fcc445a90a51e499e908f73', 'request_id': 'req-5cd3521b-48de-4409-a815-d8c2657d7965'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.043358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c617f4a84fcc445a90a51e499e908f73', 'request_id': 'req-5cd3521b-48de-4409-a815-d8c2657d7965'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.050117 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c617f4a84fcc445a90a51e499e908f73', 'request_id': 'req-5cd3521b-48de-4409-a815-d8c2657d7965'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.062888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] The token cache is being invalidate because project c617f4a84fcc445a90a51e499e908f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:43:59.063312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.063972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.065763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1242/2475] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:43:58 2022] DELETE /identity/v3/projects/c617f4a84fcc445a90a51e499e908f73 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:43:59.069832 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5b772-2098-46af-b0c4-12ed07a33539 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:59.080522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5b772-2098-46af-b0c4-12ed07a33539 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:59.142359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a291c-dfb6-4c2c-8a5a-a20abf656c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:59.143040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a291c-dfb6-4c2c-8a5a-a20abf656c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:59.143356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a291c-dfb6-4c2c-8a5a-a20abf656c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:59.144172 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a291c-dfb6-4c2c-8a5a-a20abf656c52 admin admin] PATH_INFO: `/v3/users/ba48b720e954465c8d1838ba9c320dce` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:59.145539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a291c-dfb6-4c2c-8a5a-a20abf656c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba48b720e954465c8d1838ba9c320dce)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:59.147545 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a291c-dfb6-4c2c-8a5a-a20abf656c52 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:59.212080 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-538a291c-dfb6-4c2c-8a5a-a20abf656c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba48b720e954465c8d1838ba9c320dce'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.227696 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-538a291c-dfb6-4c2c-8a5a-a20abf656c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba48b720e954465c8d1838ba9c320dce'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.228754 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-538a291c-dfb6-4c2c-8a5a-a20abf656c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba48b720e954465c8d1838ba9c320dce'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.236205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-538a291c-dfb6-4c2c-8a5a-a20abf656c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba48b720e954465c8d1838ba9c320dce'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.245123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-538a291c-dfb6-4c2c-8a5a-a20abf656c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba48b720e954465c8d1838ba9c320dce'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.254754 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1234/2476] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:43:59 2022] DELETE /identity/v3/users/ba48b720e954465c8d1838ba9c320dce => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:43:59.694104 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:43:59.696246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:59.739152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3521b-48de-4409-a815-d8c2657d7965 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:43:59.810250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ff61c-8c98-479b-8a89-1028e09aca31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:43:59.811028 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ff61c-8c98-479b-8a89-1028e09aca31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:59.811364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ff61c-8c98-479b-8a89-1028e09aca31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:59.811633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ff61c-8c98-479b-8a89-1028e09aca31 admin admin] PATH_INFO: `/v3/projects/9acf1b27daf2479c9c8ef0d2543e9369` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:59.812834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ff61c-8c98-479b-8a89-1028e09aca31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9acf1b27daf2479c9c8ef0d2543e9369)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:43:59.814764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ff61c-8c98-479b-8a89-1028e09aca31 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:43:59.840290 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:43:59.873510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cc3ff61c-8c98-479b-8a89-1028e09aca31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9acf1b27daf2479c9c8ef0d2543e9369', 'request_id': 'req-cc3ff61c-8c98-479b-8a89-1028e09aca31'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.882409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cc3ff61c-8c98-479b-8a89-1028e09aca31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9acf1b27daf2479c9c8ef0d2543e9369', 'request_id': 'req-cc3ff61c-8c98-479b-8a89-1028e09aca31'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.888255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cc3ff61c-8c98-479b-8a89-1028e09aca31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9acf1b27daf2479c9c8ef0d2543e9369', 'request_id': 'req-cc3ff61c-8c98-479b-8a89-1028e09aca31'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.899357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cc3ff61c-8c98-479b-8a89-1028e09aca31 admin admin] The token cache is being invalidate because project 9acf1b27daf2479c9c8ef0d2543e9369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:43:59.899648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cc3ff61c-8c98-479b-8a89-1028e09aca31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.900299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cc3ff61c-8c98-479b-8a89-1028e09aca31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:43:59.901826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1243/2477] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:43:59 2022] DELETE /identity/v3/projects/9acf1b27daf2479c9c8ef0d2543e9369 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:43:59.906773 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a80da-6692-4eaa-b67d-33e62fb6e230 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:43:59.907081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a80da-6692-4eaa-b67d-33e62fb6e230 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:43:59.907367 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a80da-6692-4eaa-b67d-33e62fb6e230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:43:59.927024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-182a80da-6692-4eaa-b67d-33e62fb6e230 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:43:59.928288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-182a80da-6692-4eaa-b67d-33e62fb6e230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:43:59.936864 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:00.017779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1235/2478] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:00.020421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195b45b-0138-40aa-aac0-bf5caeb63d74 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:00.020907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195b45b-0138-40aa-aac0-bf5caeb63d74 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:00.021074 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195b45b-0138-40aa-aac0-bf5caeb63d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:00.041503 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c195b45b-0138-40aa-aac0-bf5caeb63d74 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:00.042742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c195b45b-0138-40aa-aac0-bf5caeb63d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:00.051572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1244/2479] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:00.056220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a80da-6692-4eaa-b67d-33e62fb6e230 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:00.077353 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a80da-6692-4eaa-b67d-33e62fb6e230 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:00.092445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156446e-8b5c-42d8-8956-c706d55a87a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:00.093134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156446e-8b5c-42d8-8956-c706d55a87a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:00.093421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156446e-8b5c-42d8-8956-c706d55a87a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:00.094640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156446e-8b5c-42d8-8956-c706d55a87a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:00.095868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156446e-8b5c-42d8-8956-c706d55a87a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:44:00.096182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156446e-8b5c-42d8-8956-c706d55a87a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:00.098254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156446e-8b5c-42d8-8956-c706d55a87a7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:00.141987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1236/2480] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:44:00.147187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195b45b-0138-40aa-aac0-bf5caeb63d74 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:00.172347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195b45b-0138-40aa-aac0-bf5caeb63d74 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:00.186065 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c287c75-aa0c-4b4c-92a1-6e27bc0c0ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:00.186272 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c287c75-aa0c-4b4c-92a1-6e27bc0c0ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:00.186498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c287c75-aa0c-4b4c-92a1-6e27bc0c0ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:00.186777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c287c75-aa0c-4b4c-92a1-6e27bc0c0ac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:00.187595 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c287c75-aa0c-4b4c-92a1-6e27bc0c0ac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:00.192469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c287c75-aa0c-4b4c-92a1-6e27bc0c0ac8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:00.256055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1245/2481] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:44:00 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:00.256856 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156446e-8b5c-42d8-8956-c706d55a87a7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:00.272266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156446e-8b5c-42d8-8956-c706d55a87a7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:00.338922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc31c6-2736-436c-b9ae-7a9107d710dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:00.339591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc31c6-2736-436c-b9ae-7a9107d710dd admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:00.339878 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc31c6-2736-436c-b9ae-7a9107d710dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:00.340158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc31c6-2736-436c-b9ae-7a9107d710dd admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:00.340822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc31c6-2736-436c-b9ae-7a9107d710dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:00.342801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc31c6-2736-436c-b9ae-7a9107d710dd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:00.364960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1237/2482] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:44:00 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:00.368949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c287c75-aa0c-4b4c-92a1-6e27bc0c0ac8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:00.380051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c287c75-aa0c-4b4c-92a1-6e27bc0c0ac8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:00.384875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7bce7-4b50-4be5-8f90-240e30af7898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:00.385562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7bce7-4b50-4be5-8f90-240e30af7898 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:00.385918 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7bce7-4b50-4be5-8f90-240e30af7898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:00.386267 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7bce7-4b50-4be5-8f90-240e30af7898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:00.386969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7bce7-4b50-4be5-8f90-240e30af7898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:00.388591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7bce7-4b50-4be5-8f90-240e30af7898 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:00.399445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1246/2483] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:00.403423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc31c6-2736-436c-b9ae-7a9107d710dd admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:00.413701 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc31c6-2736-436c-b9ae-7a9107d710dd admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:00.418358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdd8cb-fa80-4d5f-a867-e790aedbd511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:00.418966 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdd8cb-fa80-4d5f-a867-e790aedbd511 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:00.419288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdd8cb-fa80-4d5f-a867-e790aedbd511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:00.419572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdd8cb-fa80-4d5f-a867-e790aedbd511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:00.420267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdd8cb-fa80-4d5f-a867-e790aedbd511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:00.421871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdd8cb-fa80-4d5f-a867-e790aedbd511 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:00.433350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1238/2484] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:00.437556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7bce7-4b50-4be5-8f90-240e30af7898 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:00.448453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7bce7-4b50-4be5-8f90-240e30af7898 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:00.453504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31dc68-e324-4dff-87b9-46a74879421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:00.454198 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31dc68-e324-4dff-87b9-46a74879421c admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:00.454487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31dc68-e324-4dff-87b9-46a74879421c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:00.454768 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31dc68-e324-4dff-87b9-46a74879421c admin admin] PATH_INFO: `/v3/projects/ea015c7d93bc49b9a7d0870bd5c24d69/users/522f9851d16c4f52bd8e56b3cd160833/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:00.471384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31dc68-e324-4dff-87b9-46a74879421c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea015c7d93bc49b9a7d0870bd5c24d69, user_id=522f9851d16c4f52bd8e56b3cd160833, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:00.474770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31dc68-e324-4dff-87b9-46a74879421c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:00.487699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1247/2485] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:44:00 2022] PUT /identity/v3/projects/ea015c7d93bc49b9a7d0870bd5c24d69/users/522f9851d16c4f52bd8e56b3cd160833/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:00.559318 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fb75c-ee87-4f71-be7d-a989bc3dfb35 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:00.559617 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fb75c-ee87-4f71-be7d-a989bc3dfb35 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:00.559976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fb75c-ee87-4f71-be7d-a989bc3dfb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:00.607460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-128fb75c-ee87-4f71-be7d-a989bc3dfb35 None None] MFA Rules not processed for user `522f9851d16c4f52bd8e56b3cd160833`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:00.609044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-128fb75c-ee87-4f71-be7d-a989bc3dfb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:00.702596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1239/2486] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:00.705066 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31dc68-e324-4dff-87b9-46a74879421c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:00.806637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31dc68-e324-4dff-87b9-46a74879421c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:00.812747 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea1084-8e81-461f-9c11-b0fde1c0d68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:00.813699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea1084-8e81-461f-9c11-b0fde1c0d68a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:00.814274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea1084-8e81-461f-9c11-b0fde1c0d68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:00.814794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea1084-8e81-461f-9c11-b0fde1c0d68a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:00.816472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea1084-8e81-461f-9c11-b0fde1c0d68a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:00.818798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea1084-8e81-461f-9c11-b0fde1c0d68a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:00.861676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1248/2487] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:44:00 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:00.867054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fb75c-ee87-4f71-be7d-a989bc3dfb35 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:00.887811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fb75c-ee87-4f71-be7d-a989bc3dfb35 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:00.896196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:01.032076 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7ab39-c763-4a17-a581-7dad69eb252e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:01.032721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7ab39-c763-4a17-a581-7dad69eb252e admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:01.033136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7ab39-c763-4a17-a581-7dad69eb252e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:01.033469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7ab39-c763-4a17-a581-7dad69eb252e admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:01.034851 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7ab39-c763-4a17-a581-7dad69eb252e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:01.036954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7ab39-c763-4a17-a581-7dad69eb252e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:01.064787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1240/2488] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:44:00 2022] POST /identity/v3/users => generated 299 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:01.069320 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea1084-8e81-461f-9c11-b0fde1c0d68a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:01.084390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea1084-8e81-461f-9c11-b0fde1c0d68a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:01.098248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d8456-67c7-46d3-b255-ffddf2365d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:01.099388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8456-67c7-46d3-b255-ffddf2365d80 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:01.099862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8456-67c7-46d3-b255-ffddf2365d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:01.100147 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8456-67c7-46d3-b255-ffddf2365d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:01.101044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d8456-67c7-46d3-b255-ffddf2365d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:01.106564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d8456-67c7-46d3-b255-ffddf2365d80 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:01.131257 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1249/2489] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:01.135127 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7ab39-c763-4a17-a581-7dad69eb252e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:01.146596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7ab39-c763-4a17-a581-7dad69eb252e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:01.151784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb30d6f-3fcb-4145-83ba-224bee184f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:01.152551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb30d6f-3fcb-4145-83ba-224bee184f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:01.152846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb30d6f-3fcb-4145-83ba-224bee184f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:01.153130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb30d6f-3fcb-4145-83ba-224bee184f2e admin admin] PATH_INFO: `/v3/projects/13b949b49d6a4f54b4d4d29bbcf0eeb1/users/0bea5987a52b444b92259b45df26519e/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:01.171155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb30d6f-3fcb-4145-83ba-224bee184f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b949b49d6a4f54b4d4d29bbcf0eeb1, user_id=0bea5987a52b444b92259b45df26519e, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:01.173893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb30d6f-3fcb-4145-83ba-224bee184f2e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:01.192405 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1241/2490] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:44:01 2022] PUT /identity/v3/projects/13b949b49d6a4f54b4d4d29bbcf0eeb1/users/0bea5987a52b444b92259b45df26519e/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:44:01.196493 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d8456-67c7-46d3-b255-ffddf2365d80 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:01.207159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d8456-67c7-46d3-b255-ffddf2365d80 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:01.219387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:01.261419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb30d6f-3fcb-4145-83ba-224bee184f2e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:01.263823 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-fdb30d6f-3fcb-4145-83ba-224bee184f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:01.290455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d898fe0-dfe8-4395-83b9-3a51a1c0b816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:01.291327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d898fe0-dfe8-4395-83b9-3a51a1c0b816 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:01.291852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d898fe0-dfe8-4395-83b9-3a51a1c0b816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:01.292606 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d898fe0-dfe8-4395-83b9-3a51a1c0b816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:01.293370 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d898fe0-dfe8-4395-83b9-3a51a1c0b816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:01.294979 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d898fe0-dfe8-4395-83b9-3a51a1c0b816 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:01.309902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1250/2491] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:01.322282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d898fe0-dfe8-4395-83b9-3a51a1c0b816 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:01.332709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d898fe0-dfe8-4395-83b9-3a51a1c0b816 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:01.337763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74b989-755b-4190-8340-9a7b09cdb2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:01.338442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74b989-755b-4190-8340-9a7b09cdb2b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:01.338731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74b989-755b-4190-8340-9a7b09cdb2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:01.339018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74b989-755b-4190-8340-9a7b09cdb2b7 admin admin] PATH_INFO: `/v3/projects/13b949b49d6a4f54b4d4d29bbcf0eeb1/users/0bea5987a52b444b92259b45df26519e/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:01.341055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74b989-755b-4190-8340-9a7b09cdb2b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b949b49d6a4f54b4d4d29bbcf0eeb1, user_id=0bea5987a52b444b92259b45df26519e, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:01.343776 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74b989-755b-4190-8340-9a7b09cdb2b7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:01.349793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb30d6f-3fcb-4145-83ba-224bee184f2e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:01.363789 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1251/2492] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:44:01 2022] PUT /identity/v3/projects/13b949b49d6a4f54b4d4d29bbcf0eeb1/users/0bea5987a52b444b92259b45df26519e/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:44:01.374044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a03b0-1f17-4c7a-8e9d-447b3990c6b9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:01.374340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a03b0-1f17-4c7a-8e9d-447b3990c6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:01.375179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a03b0-1f17-4c7a-8e9d-447b3990c6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:01.407425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-492a03b0-1f17-4c7a-8e9d-447b3990c6b9 None None] MFA Rules not processed for user `0bea5987a52b444b92259b45df26519e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:01.408746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-492a03b0-1f17-4c7a-8e9d-447b3990c6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:01.423990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:01.424656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:01.424985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:01.425277 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] PATH_INFO: `/v3/users/6e461e65817648b0a878d8280c84dc6e` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:01.426467 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e461e65817648b0a878d8280c84dc6e)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:01.428447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:01.457299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:01.484858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e461e65817648b0a878d8280c84dc6e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:01.488861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1252/2493] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:01.496008 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a03b0-1f17-4c7a-8e9d-447b3990c6b9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:01.497233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e461e65817648b0a878d8280c84dc6e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:01.497377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-492a03b0-1f17-4c7a-8e9d-447b3990c6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:01.498231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e461e65817648b0a878d8280c84dc6e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:01.506186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e461e65817648b0a878d8280c84dc6e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:01.516022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e461e65817648b0a878d8280c84dc6e'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:01.524903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1242/2494] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:44:01 2022] DELETE /identity/v3/users/6e461e65817648b0a878d8280c84dc6e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:01.563189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a03b0-1f17-4c7a-8e9d-447b3990c6b9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:01.604769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:01.616502 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b3f2b-77df-42ea-8046-25dda81ecaa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:01.617128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b3f2b-77df-42ea-8046-25dda81ecaa4 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:01.617420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b3f2b-77df-42ea-8046-25dda81ecaa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:01.617825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b3f2b-77df-42ea-8046-25dda81ecaa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:01.619733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e35b3f2b-77df-42ea-8046-25dda81ecaa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:01.684029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b3f2b-77df-42ea-8046-25dda81ecaa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:01.685542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b3f2b-77df-42ea-8046-25dda81ecaa4 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:01.698750 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1253/2495] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:44:01.705163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:01.707507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:01.793070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945658f9-5086-4594-8625-61a02e0cd034 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:01.798443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:01.799908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:01.801421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:01.801893 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] PATH_INFO: `/v3/projects/a27abd6ed13c4174a4c14226fb48e146` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:01.803363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] RBAC: Authorizing `identity:delete_project(project_id=a27abd6ed13c4174a4c14226fb48e146)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:01.805458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:01.867337 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a27abd6ed13c4174a4c14226fb48e146', 'request_id': 'req-25a3a452-078f-4894-98b7-4b14424dedef'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:01.874251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a27abd6ed13c4174a4c14226fb48e146', 'request_id': 'req-25a3a452-078f-4894-98b7-4b14424dedef'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:01.887137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a27abd6ed13c4174a4c14226fb48e146', 'request_id': 'req-25a3a452-078f-4894-98b7-4b14424dedef'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:01.910149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] The token cache is being invalidate because project a27abd6ed13c4174a4c14226fb48e146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:44:01.910510 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:01.911168 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:01.913741 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1243/2496] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:44:01 2022] DELETE /identity/v3/projects/a27abd6ed13c4174a4c14226fb48e146 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:01.918256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b3f2b-77df-42ea-8046-25dda81ecaa4 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:01.935630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b3f2b-77df-42ea-8046-25dda81ecaa4 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:02.029975 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:02.030665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:02.030955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:02.031260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] PATH_INFO: `/v3/users/d1237c9f979541feb603ee86b9cb3536` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:02.032558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1237c9f979541feb603ee86b9cb3536)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:02.035623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:02.103316 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1237c9f979541feb603ee86b9cb3536'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.112992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1237c9f979541feb603ee86b9cb3536'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.133181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1237c9f979541feb603ee86b9cb3536'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.133972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1237c9f979541feb603ee86b9cb3536'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.134619 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1237c9f979541feb603ee86b9cb3536'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.163392 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1254/2497] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:44:01 2022] DELETE /identity/v3/users/d1237c9f979541feb603ee86b9cb3536 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:44:02.395548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:02.397879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:02.420282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3a452-078f-4894-98b7-4b14424dedef admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:02.496482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:02.552803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:02.554276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:02.584951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:02.601742 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ac37a-94bb-463b-8d61-196c2dcd0d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:02.602892 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ac37a-94bb-463b-8d61-196c2dcd0d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:02.603218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ac37a-94bb-463b-8d61-196c2dcd0d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:02.603503 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ac37a-94bb-463b-8d61-196c2dcd0d5d admin admin] PATH_INFO: `/v3/projects/7c3a677ae42a49538832ba7e24489ff8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:02.604905 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ac37a-94bb-463b-8d61-196c2dcd0d5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c3a677ae42a49538832ba7e24489ff8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:02.607464 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ac37a-94bb-463b-8d61-196c2dcd0d5d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:02.648708 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a517cc-c5d2-4de9-a6d2-fcd51d098ac2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:02.653804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:02.654447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:02.654898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:02.655218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] PATH_INFO: `/v3/users/522f9851d16c4f52bd8e56b3cd160833` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:02.656396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=522f9851d16c4f52bd8e56b3cd160833)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:02.658318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:02.670969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-018ac37a-94bb-463b-8d61-196c2dcd0d5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c3a677ae42a49538832ba7e24489ff8', 'request_id': 'req-018ac37a-94bb-463b-8d61-196c2dcd0d5d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.679600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-018ac37a-94bb-463b-8d61-196c2dcd0d5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c3a677ae42a49538832ba7e24489ff8', 'request_id': 'req-018ac37a-94bb-463b-8d61-196c2dcd0d5d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.689160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-018ac37a-94bb-463b-8d61-196c2dcd0d5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c3a677ae42a49538832ba7e24489ff8', 'request_id': 'req-018ac37a-94bb-463b-8d61-196c2dcd0d5d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.706396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-018ac37a-94bb-463b-8d61-196c2dcd0d5d admin admin] The token cache is being invalidate because project 7c3a677ae42a49538832ba7e24489ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:44:02.706896 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-018ac37a-94bb-463b-8d61-196c2dcd0d5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.708034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-018ac37a-94bb-463b-8d61-196c2dcd0d5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.709836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1244/2498] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:44:02 2022] DELETE /identity/v3/projects/7c3a677ae42a49538832ba7e24489ff8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:02.713134 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '522f9851d16c4f52bd8e56b3cd160833'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.721408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '522f9851d16c4f52bd8e56b3cd160833'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.736898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '522f9851d16c4f52bd8e56b3cd160833'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.738888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '522f9851d16c4f52bd8e56b3cd160833'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.739481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '522f9851d16c4f52bd8e56b3cd160833'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:02.754368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1255/2499] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:44:02 2022] DELETE /identity/v3/users/522f9851d16c4f52bd8e56b3cd160833 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:02.759107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f795f6a-baf2-4f7a-9087-dfb5fb9e7d03 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:02.759412 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f795f6a-baf2-4f7a-9087-dfb5fb9e7d03 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:02.759691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f795f6a-baf2-4f7a-9087-dfb5fb9e7d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:02.782185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6f795f6a-baf2-4f7a-9087-dfb5fb9e7d03 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:02.783471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6f795f6a-baf2-4f7a-9087-dfb5fb9e7d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:02.848487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1245/2500] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:02.858512 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:02.860071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:02.928544 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d7918-3d6e-45db-8709-00b9f2c76c99 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:02.932664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153a196-8dcd-4ec8-811e-f3c7c61697a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:02.933311 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153a196-8dcd-4ec8-811e-f3c7c61697a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:02.933600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153a196-8dcd-4ec8-811e-f3c7c61697a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:02.933913 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153a196-8dcd-4ec8-811e-f3c7c61697a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:02.945420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153a196-8dcd-4ec8-811e-f3c7c61697a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:02.946916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153a196-8dcd-4ec8-811e-f3c7c61697a0 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:02.958179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1256/2501] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:03.235903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f795f6a-baf2-4f7a-9087-dfb5fb9e7d03 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:03.238174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6f795f6a-baf2-4f7a-9087-dfb5fb9e7d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:03.252422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f795f6a-baf2-4f7a-9087-dfb5fb9e7d03 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:03.258204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:03.259305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:03.259305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:03.259661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] PATH_INFO: `/v3/projects/ea015c7d93bc49b9a7d0870bd5c24d69` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:03.260925 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea015c7d93bc49b9a7d0870bd5c24d69)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:03.263274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:03.313014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea015c7d93bc49b9a7d0870bd5c24d69', 'request_id': 'req-c1b70933-ccff-460a-9ae7-15247ace9ccf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:03.317982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:03.319346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea015c7d93bc49b9a7d0870bd5c24d69', 'request_id': 'req-c1b70933-ccff-460a-9ae7-15247ace9ccf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:03.326985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea015c7d93bc49b9a7d0870bd5c24d69', 'request_id': 'req-c1b70933-ccff-460a-9ae7-15247ace9ccf'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:03.339752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] The token cache is being invalidate because project ea015c7d93bc49b9a7d0870bd5c24d69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:44:03.340042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:03.340654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:03.342205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1246/2502] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:44:03 2022] DELETE /identity/v3/projects/ea015c7d93bc49b9a7d0870bd5c24d69 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:03.346348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153a196-8dcd-4ec8-811e-f3c7c61697a0 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:03.356263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153a196-8dcd-4ec8-811e-f3c7c61697a0 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:03.388011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:03.416950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7e0ab-3d0d-47f9-835c-a53a4d1db036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:03.417637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7e0ab-3d0d-47f9-835c-a53a4d1db036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:03.417922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7e0ab-3d0d-47f9-835c-a53a4d1db036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:03.418197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7e0ab-3d0d-47f9-835c-a53a4d1db036 admin admin] PATH_INFO: `/v3/users/0bea5987a52b444b92259b45df26519e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:03.419396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7e0ab-3d0d-47f9-835c-a53a4d1db036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bea5987a52b444b92259b45df26519e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:03.421303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7e0ab-3d0d-47f9-835c-a53a4d1db036 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:03.465286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1cf7e0ab-3d0d-47f9-835c-a53a4d1db036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bea5987a52b444b92259b45df26519e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:03.473094 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1cf7e0ab-3d0d-47f9-835c-a53a4d1db036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bea5987a52b444b92259b45df26519e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:03.484857 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1cf7e0ab-3d0d-47f9-835c-a53a4d1db036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bea5987a52b444b92259b45df26519e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:03.485442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1cf7e0ab-3d0d-47f9-835c-a53a4d1db036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bea5987a52b444b92259b45df26519e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:03.486025 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1cf7e0ab-3d0d-47f9-835c-a53a4d1db036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bea5987a52b444b92259b45df26519e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:03.498452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1257/2503] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:44:03 2022] DELETE /identity/v3/users/0bea5987a52b444b92259b45df26519e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:03.807352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:03.808987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:03.825001 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b70933-ccff-460a-9ae7-15247ace9ccf admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:03.907484 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46beba65-e787-4804-8068-85d61188ebc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:03.908095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46beba65-e787-4804-8068-85d61188ebc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:03.908454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46beba65-e787-4804-8068-85d61188ebc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:03.908811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46beba65-e787-4804-8068-85d61188ebc8 admin admin] PATH_INFO: `/v3/projects/13b949b49d6a4f54b4d4d29bbcf0eeb1` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:03.912525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46beba65-e787-4804-8068-85d61188ebc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13b949b49d6a4f54b4d4d29bbcf0eeb1)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:03.914601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46beba65-e787-4804-8068-85d61188ebc8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:03.978774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-46beba65-e787-4804-8068-85d61188ebc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13b949b49d6a4f54b4d4d29bbcf0eeb1', 'request_id': 'req-46beba65-e787-4804-8068-85d61188ebc8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:03.989325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-46beba65-e787-4804-8068-85d61188ebc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13b949b49d6a4f54b4d4d29bbcf0eeb1', 'request_id': 'req-46beba65-e787-4804-8068-85d61188ebc8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:03.997866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-46beba65-e787-4804-8068-85d61188ebc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13b949b49d6a4f54b4d4d29bbcf0eeb1', 'request_id': 'req-46beba65-e787-4804-8068-85d61188ebc8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:04.013868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-46beba65-e787-4804-8068-85d61188ebc8 admin admin] The token cache is being invalidate because project 13b949b49d6a4f54b4d4d29bbcf0eeb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:44:04.014385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-46beba65-e787-4804-8068-85d61188ebc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:04.015175 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-46beba65-e787-4804-8068-85d61188ebc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:04.020200 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1247/2504] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:44:03 2022] DELETE /identity/v3/projects/13b949b49d6a4f54b4d4d29bbcf0eeb1 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:04.021795 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0723d5d-f0fd-4a30-bbc6-f5a9a20ff483 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:04.022084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0723d5d-f0fd-4a30-bbc6-f5a9a20ff483 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:04.022364 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0723d5d-f0fd-4a30-bbc6-f5a9a20ff483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:04.057023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d0723d5d-f0fd-4a30-bbc6-f5a9a20ff483 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:04.059020 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d0723d5d-f0fd-4a30-bbc6-f5a9a20ff483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:04.204235 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1258/2505] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:04.222143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe2c26-17ce-43b2-8991-b74de16bdda3 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:04.222426 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe2c26-17ce-43b2-8991-b74de16bdda3 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:04.222713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe2c26-17ce-43b2-8991-b74de16bdda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:04.256289 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-28fe2c26-17ce-43b2-8991-b74de16bdda3 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:04.257581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-28fe2c26-17ce-43b2-8991-b74de16bdda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:04.265923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1248/2506] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:04.270815 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0723d5d-f0fd-4a30-bbc6-f5a9a20ff483 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:04.287345 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0723d5d-f0fd-4a30-bbc6-f5a9a20ff483 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:04.292995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c075a4-b8ca-40e0-9b94-0e8bf8837399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:04.293682 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c075a4-b8ca-40e0-9b94-0e8bf8837399 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:04.293999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c075a4-b8ca-40e0-9b94-0e8bf8837399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:04.294341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c075a4-b8ca-40e0-9b94-0e8bf8837399 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:04.295266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c075a4-b8ca-40e0-9b94-0e8bf8837399 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:44:04.295559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c075a4-b8ca-40e0-9b94-0e8bf8837399 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:04.297373 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c075a4-b8ca-40e0-9b94-0e8bf8837399 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:04.310329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1259/2507] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:44:04.314706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe2c26-17ce-43b2-8991-b74de16bdda3 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:04.325139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe2c26-17ce-43b2-8991-b74de16bdda3 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:04.329830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a6bf9-f18d-4c75-92a0-5bde3c715e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:04.330426 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a6bf9-f18d-4c75-92a0-5bde3c715e8a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:04.330718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a6bf9-f18d-4c75-92a0-5bde3c715e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:04.331426 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a6bf9-f18d-4c75-92a0-5bde3c715e8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:04.332160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a6bf9-f18d-4c75-92a0-5bde3c715e8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:04.334083 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a6bf9-f18d-4c75-92a0-5bde3c715e8a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:04.365382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1249/2508] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:44:04 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:04.370165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c075a4-b8ca-40e0-9b94-0e8bf8837399 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:04.381468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c075a4-b8ca-40e0-9b94-0e8bf8837399 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:04.422279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:04.446725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1ffed-eed5-48c0-93b4-6ec111164353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:04.447390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1ffed-eed5-48c0-93b4-6ec111164353 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:04.447709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1ffed-eed5-48c0-93b4-6ec111164353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:04.448016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1ffed-eed5-48c0-93b4-6ec111164353 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:04.448738 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1ffed-eed5-48c0-93b4-6ec111164353 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:04.450718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1ffed-eed5-48c0-93b4-6ec111164353 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:04.472112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1260/2509] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:44:04 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:04.476119 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a6bf9-f18d-4c75-92a0-5bde3c715e8a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:04.486252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a6bf9-f18d-4c75-92a0-5bde3c715e8a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:04.491867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88394e7-5afe-436a-ae31-5c8354893eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:04.492502 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88394e7-5afe-436a-ae31-5c8354893eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:04.492780 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88394e7-5afe-436a-ae31-5c8354893eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:04.493058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88394e7-5afe-436a-ae31-5c8354893eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:04.493748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88394e7-5afe-436a-ae31-5c8354893eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:04.495446 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88394e7-5afe-436a-ae31-5c8354893eb9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:04.506309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1250/2510] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:04.510185 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1ffed-eed5-48c0-93b4-6ec111164353 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:04.520553 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1ffed-eed5-48c0-93b4-6ec111164353 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:04.525208 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665d38f-e0cd-4f90-a1d2-851b1117c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:04.525811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665d38f-e0cd-4f90-a1d2-851b1117c962 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:04.526097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665d38f-e0cd-4f90-a1d2-851b1117c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:04.526373 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665d38f-e0cd-4f90-a1d2-851b1117c962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:04.527057 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665d38f-e0cd-4f90-a1d2-851b1117c962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:04.528677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665d38f-e0cd-4f90-a1d2-851b1117c962 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:04.539283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1261/2511] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:04.543324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88394e7-5afe-436a-ae31-5c8354893eb9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:04.553779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88394e7-5afe-436a-ae31-5c8354893eb9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:04.558364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ec061-72e3-40cc-90a5-6885307d71a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:04.559017 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ec061-72e3-40cc-90a5-6885307d71a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:04.559331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ec061-72e3-40cc-90a5-6885307d71a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:04.559610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ec061-72e3-40cc-90a5-6885307d71a3 admin admin] PATH_INFO: `/v3/projects/b30509605c224f8997e09243625b16fd/users/60bb92b0397c403f873bf359b79b2a82/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:04.699035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ec061-72e3-40cc-90a5-6885307d71a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30509605c224f8997e09243625b16fd, user_id=60bb92b0397c403f873bf359b79b2a82, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:04.701706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ec061-72e3-40cc-90a5-6885307d71a3 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:04.723813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1251/2512] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:44:04 2022] PUT /identity/v3/projects/b30509605c224f8997e09243625b16fd/users/60bb92b0397c403f873bf359b79b2a82/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:04.725547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fd88d-6a18-4a16-890f-75d24abdf2fb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:04.725869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fd88d-6a18-4a16-890f-75d24abdf2fb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:04.726154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fd88d-6a18-4a16-890f-75d24abdf2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:04.754699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9e2fd88d-6a18-4a16-890f-75d24abdf2fb None None] MFA Rules not processed for user `60bb92b0397c403f873bf359b79b2a82`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:04.755867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9e2fd88d-6a18-4a16-890f-75d24abdf2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:04.821796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1262/2513] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:04.826247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ec061-72e3-40cc-90a5-6885307d71a3 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:04.838779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:04.889819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ec061-72e3-40cc-90a5-6885307d71a3 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:04.894426 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e29d7-9fb3-45ed-be43-7a33cabc0312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:04.895029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e29d7-9fb3-45ed-be43-7a33cabc0312 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:04.895341 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e29d7-9fb3-45ed-be43-7a33cabc0312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:04.895614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e29d7-9fb3-45ed-be43-7a33cabc0312 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:04.896279 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e29d7-9fb3-45ed-be43-7a33cabc0312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:04.898158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e29d7-9fb3-45ed-be43-7a33cabc0312 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:04.924065 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1252/2514] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:44:04 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:04.928543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fd88d-6a18-4a16-890f-75d24abdf2fb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:04.939009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fd88d-6a18-4a16-890f-75d24abdf2fb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:05.004666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1d146-80ee-482a-b1cc-5a0502e5ef07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:05.005362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1d146-80ee-482a-b1cc-5a0502e5ef07 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:05.005745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1d146-80ee-482a-b1cc-5a0502e5ef07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:05.006031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1d146-80ee-482a-b1cc-5a0502e5ef07 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:05.006734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1d146-80ee-482a-b1cc-5a0502e5ef07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:05.008703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1d146-80ee-482a-b1cc-5a0502e5ef07 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:05.031847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1263/2515] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:44:04 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:44:05.036571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e29d7-9fb3-45ed-be43-7a33cabc0312 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:05.046864 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e29d7-9fb3-45ed-be43-7a33cabc0312 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:05.052360 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cb053-6f89-436e-80ac-4170d29e1d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:05.053067 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cb053-6f89-436e-80ac-4170d29e1d9f admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:05.053420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cb053-6f89-436e-80ac-4170d29e1d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:05.053762 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cb053-6f89-436e-80ac-4170d29e1d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:05.054521 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cb053-6f89-436e-80ac-4170d29e1d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:05.056235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cb053-6f89-436e-80ac-4170d29e1d9f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:05.067339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1253/2516] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:05.071386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1d146-80ee-482a-b1cc-5a0502e5ef07 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:05.082313 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1d146-80ee-482a-b1cc-5a0502e5ef07 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:05.087108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344456bb-4a3e-4d34-acbf-ae7123db6fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:05.087774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344456bb-4a3e-4d34-acbf-ae7123db6fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:05.088090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344456bb-4a3e-4d34-acbf-ae7123db6fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:05.088370 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344456bb-4a3e-4d34-acbf-ae7123db6fe7 admin admin] PATH_INFO: `/v3/projects/18d2ab2c9ded4bfdbdd8d65d52aad035/users/51bed19553004a728b04c0b359c8ac57/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:05.104408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344456bb-4a3e-4d34-acbf-ae7123db6fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18d2ab2c9ded4bfdbdd8d65d52aad035, user_id=51bed19553004a728b04c0b359c8ac57, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:05.107035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344456bb-4a3e-4d34-acbf-ae7123db6fe7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:05.123018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1264/2517] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:44:05 2022] PUT /identity/v3/projects/18d2ab2c9ded4bfdbdd8d65d52aad035/users/51bed19553004a728b04c0b359c8ac57/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:05.127382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cb053-6f89-436e-80ac-4170d29e1d9f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:05.137486 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cb053-6f89-436e-80ac-4170d29e1d9f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:05.194037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c48b5-c00f-4949-88f3-144396c2db3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:05.194672 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c48b5-c00f-4949-88f3-144396c2db3a admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:05.194955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c48b5-c00f-4949-88f3-144396c2db3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:05.195269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c48b5-c00f-4949-88f3-144396c2db3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:05.195955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c48b5-c00f-4949-88f3-144396c2db3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:05.197592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c48b5-c00f-4949-88f3-144396c2db3a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:05.207928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1254/2518] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:05.212132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344456bb-4a3e-4d34-acbf-ae7123db6fe7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:05.223195 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344456bb-4a3e-4d34-acbf-ae7123db6fe7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:05.227835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d032318-8f3e-4847-a6cc-b256da81a104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:05.228488 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d032318-8f3e-4847-a6cc-b256da81a104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:05.228777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d032318-8f3e-4847-a6cc-b256da81a104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:05.229059 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d032318-8f3e-4847-a6cc-b256da81a104 admin admin] PATH_INFO: `/v3/projects/18d2ab2c9ded4bfdbdd8d65d52aad035/users/51bed19553004a728b04c0b359c8ac57/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:05.231021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d032318-8f3e-4847-a6cc-b256da81a104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18d2ab2c9ded4bfdbdd8d65d52aad035, user_id=51bed19553004a728b04c0b359c8ac57, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:05.233680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d032318-8f3e-4847-a6cc-b256da81a104 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:05.239974 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:05.245361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1265/2519] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:44:05 2022] PUT /identity/v3/projects/18d2ab2c9ded4bfdbdd8d65d52aad035/users/51bed19553004a728b04c0b359c8ac57/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:05.255840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed9136-7315-4ef0-9e08-33ce09d85f14 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:05.256068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed9136-7315-4ef0-9e08-33ce09d85f14 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:05.256345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed9136-7315-4ef0-9e08-33ce09d85f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:05.283868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d9ed9136-7315-4ef0-9e08-33ce09d85f14 None None] MFA Rules not processed for user `51bed19553004a728b04c0b359c8ac57`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:05.286348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d9ed9136-7315-4ef0-9e08-33ce09d85f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:05.354698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1255/2520] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:05.366179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d032318-8f3e-4847-a6cc-b256da81a104 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:05.367885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8d032318-8f3e-4847-a6cc-b256da81a104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:05.424901 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d032318-8f3e-4847-a6cc-b256da81a104 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:05.428664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f1968-9a2f-4420-9d1c-148e48bc3901 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:05.429333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f1968-9a2f-4420-9d1c-148e48bc3901 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:05.429619 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f1968-9a2f-4420-9d1c-148e48bc3901 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:05.429897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f1968-9a2f-4420-9d1c-148e48bc3901 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:05.438742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f1968-9a2f-4420-9d1c-148e48bc3901 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:05.440773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f1968-9a2f-4420-9d1c-148e48bc3901 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:05.450377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1266/2521] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:05.541348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed9136-7315-4ef0-9e08-33ce09d85f14 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:05.551101 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed9136-7315-4ef0-9e08-33ce09d85f14 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:05.554680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c27fd-955a-4217-b719-605071777e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:05.555328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c27fd-955a-4217-b719-605071777e80 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:05.555611 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c27fd-955a-4217-b719-605071777e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:05.555886 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c27fd-955a-4217-b719-605071777e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:05.560219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:05.616585 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c27fd-955a-4217-b719-605071777e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:05.618041 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c27fd-955a-4217-b719-605071777e80 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:05.628388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1256/2522] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:05.893907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f1968-9a2f-4420-9d1c-148e48bc3901 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:05.956513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f1968-9a2f-4420-9d1c-148e48bc3901 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:05.961123 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:05.961764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:05.962049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:05.962330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] PATH_INFO: `/v3/users/60bb92b0397c403f873bf359b79b2a82` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:05.963524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=60bb92b0397c403f873bf359b79b2a82)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:05.965498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:06.012189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60bb92b0397c403f873bf359b79b2a82'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:06.024011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60bb92b0397c403f873bf359b79b2a82'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:06.037409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60bb92b0397c403f873bf359b79b2a82'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:06.038132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60bb92b0397c403f873bf359b79b2a82'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:06.038729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60bb92b0397c403f873bf359b79b2a82'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:06.051791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1267/2523] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:44:05 2022] DELETE /identity/v3/users/60bb92b0397c403f873bf359b79b2a82 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:06.056370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875e16b-70ce-4cb9-8fcf-89de5d277a79 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:06.056830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875e16b-70ce-4cb9-8fcf-89de5d277a79 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:06.057020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875e16b-70ce-4cb9-8fcf-89de5d277a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:06.076641 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8875e16b-70ce-4cb9-8fcf-89de5d277a79 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:06.077845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8875e16b-70ce-4cb9-8fcf-89de5d277a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:06.139590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1257/2524] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:06.146234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:06.147816 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:06.175356 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:06.208916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ecaf3-d3ea-4ed3-a5b5-0bc61edc4bae admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:06.212567 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f490f1-c57d-469c-8be8-4aedebde3248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:06.213181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f490f1-c57d-469c-8be8-4aedebde3248 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:06.213459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f490f1-c57d-469c-8be8-4aedebde3248 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:06.213735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f490f1-c57d-469c-8be8-4aedebde3248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:06.223533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f490f1-c57d-469c-8be8-4aedebde3248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:06.224967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f490f1-c57d-469c-8be8-4aedebde3248 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:06.235274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1268/2525] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:06.785110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875e16b-70ce-4cb9-8fcf-89de5d277a79 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:06.786626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8875e16b-70ce-4cb9-8fcf-89de5d277a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:06.801359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875e16b-70ce-4cb9-8fcf-89de5d277a79 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:06.805934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:06.806577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:06.806864 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:06.807174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] PATH_INFO: `/v3/projects/b30509605c224f8997e09243625b16fd` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:06.808338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30509605c224f8997e09243625b16fd)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:06.810249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:06.867586 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30509605c224f8997e09243625b16fd', 'request_id': 'req-1d7929d6-85a5-46c0-9dc2-dd15df63f143'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:06.873286 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30509605c224f8997e09243625b16fd', 'request_id': 'req-1d7929d6-85a5-46c0-9dc2-dd15df63f143'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:06.880992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30509605c224f8997e09243625b16fd', 'request_id': 'req-1d7929d6-85a5-46c0-9dc2-dd15df63f143'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:06.894188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] The token cache is being invalidate because project b30509605c224f8997e09243625b16fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:44:06.894482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:06.895151 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:06.896693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1258/2526] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:44:06 2022] DELETE /identity/v3/projects/b30509605c224f8997e09243625b16fd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:06.900751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f490f1-c57d-469c-8be8-4aedebde3248 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:06.910951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f490f1-c57d-469c-8be8-4aedebde3248 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:06.975967 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29633c-b477-411c-a5e8-69c0bb483dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:06.976595 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29633c-b477-411c-a5e8-69c0bb483dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:06.976910 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29633c-b477-411c-a5e8-69c0bb483dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:06.977849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29633c-b477-411c-a5e8-69c0bb483dce admin admin] PATH_INFO: `/v3/users/51bed19553004a728b04c0b359c8ac57` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:06.978464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29633c-b477-411c-a5e8-69c0bb483dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=51bed19553004a728b04c0b359c8ac57)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:06.980488 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29633c-b477-411c-a5e8-69c0bb483dce admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:07.027431 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0d29633c-b477-411c-a5e8-69c0bb483dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51bed19553004a728b04c0b359c8ac57'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:07.035749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0d29633c-b477-411c-a5e8-69c0bb483dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51bed19553004a728b04c0b359c8ac57'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:07.053429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0d29633c-b477-411c-a5e8-69c0bb483dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51bed19553004a728b04c0b359c8ac57'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:07.054445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0d29633c-b477-411c-a5e8-69c0bb483dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51bed19553004a728b04c0b359c8ac57'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:07.055460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0d29633c-b477-411c-a5e8-69c0bb483dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51bed19553004a728b04c0b359c8ac57'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:07.070538 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1269/2527] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:44:06 2022] DELETE /identity/v3/users/51bed19553004a728b04c0b359c8ac57 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:07.368155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:07.374520 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:07.415373 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7929d6-85a5-46c0-9dc2-dd15df63f143 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:07.457361 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:07.511307 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:07.514280 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:07.514766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:07.515299 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25 admin admin] PATH_INFO: `/v3/projects/18d2ab2c9ded4bfdbdd8d65d52aad035` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:07.517793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18d2ab2c9ded4bfdbdd8d65d52aad035)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:07.520692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:07.585210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18d2ab2c9ded4bfdbdd8d65d52aad035', 'request_id': 'req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:07.597259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18d2ab2c9ded4bfdbdd8d65d52aad035', 'request_id': 'req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:07.600832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d57c24-5cb3-40c7-8b08-ff9285a80b84 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:07.601223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d57c24-5cb3-40c7-8b08-ff9285a80b84 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:07.601662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d57c24-5cb3-40c7-8b08-ff9285a80b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:07.604231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18d2ab2c9ded4bfdbdd8d65d52aad035', 'request_id': 'req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:07.629957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-64d57c24-5cb3-40c7-8b08-ff9285a80b84 None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:07.629957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-64d57c24-5cb3-40c7-8b08-ff9285a80b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:07.633127 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25 admin admin] The token cache is being invalidate because project 18d2ab2c9ded4bfdbdd8d65d52aad035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:44:07.633618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:07.634534 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0d8479d8-05bf-4e2f-9c3b-cebc27c3db25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:07.636366 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1259/2528] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:44:07 2022] DELETE /identity/v3/projects/18d2ab2c9ded4bfdbdd8d65d52aad035 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:07.641285 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a29e-d0cc-4050-ba63-eb2cdf131a4c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:07.641585 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a29e-d0cc-4050-ba63-eb2cdf131a4c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:07.641906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a29e-d0cc-4050-ba63-eb2cdf131a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:07.683849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e370a29e-d0cc-4050-ba63-eb2cdf131a4c None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:07.685396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e370a29e-d0cc-4050-ba63-eb2cdf131a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:07.686154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:07.720692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1270/2529] 10.222.0.40 () {58 vars in 967 bytes} [Mon Aug 1 07:44:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:07.737675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d57c24-5cb3-40c7-8b08-ff9285a80b84 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:07.739388 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-64d57c24-5cb3-40c7-8b08-ff9285a80b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:07.775368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1260/2530] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:44:07.792565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8931e-df73-488a-b7f1-ff7f34d9763b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:07.792860 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8931e-df73-488a-b7f1-ff7f34d9763b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:07.793141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8931e-df73-488a-b7f1-ff7f34d9763b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:07.815620 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-eaf8931e-df73-488a-b7f1-ff7f34d9763b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:07.817402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-eaf8931e-df73-488a-b7f1-ff7f34d9763b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:07.827096 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1261/2531] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:07.831009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8931e-df73-488a-b7f1-ff7f34d9763b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:07.833398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d57c24-5cb3-40c7-8b08-ff9285a80b84 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:07.840266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648a6b5-f51f-4099-ab9c-eea15c6d358b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:07.840552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648a6b5-f51f-4099-ab9c-eea15c6d358b service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:07.840842 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648a6b5-f51f-4099-ab9c-eea15c6d358b service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:07.840950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8931e-df73-488a-b7f1-ff7f34d9763b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:07.841126 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648a6b5-f51f-4099-ab9c-eea15c6d358b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:07.847550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab19f9-f100-4a4d-b609-fc21357dc15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:07.848748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab19f9-f100-4a4d-b609-fc21357dc15f admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:07.848748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab19f9-f100-4a4d-b609-fc21357dc15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:07.849194 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab19f9-f100-4a4d-b609-fc21357dc15f admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:07.849996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab19f9-f100-4a4d-b609-fc21357dc15f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:44:07.850385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab19f9-f100-4a4d-b609-fc21357dc15f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:07.851112 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648a6b5-f51f-4099-ab9c-eea15c6d358b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:07.852220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab19f9-f100-4a4d-b609-fc21357dc15f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:07.852346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648a6b5-f51f-4099-ab9c-eea15c6d358b service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:07.861662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1271/2532] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:44:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 07:44:07.871191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1262/2533] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:44:07.872396 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648a6b5-f51f-4099-ab9c-eea15c6d358b service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:07.883919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648a6b5-f51f-4099-ab9c-eea15c6d358b service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:07.889896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed392d46-bf44-47ce-9530-e0f6ff2cae12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:07.890690 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed392d46-bf44-47ce-9530-e0f6ff2cae12 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:07.891038 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed392d46-bf44-47ce-9530-e0f6ff2cae12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:07.891412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed392d46-bf44-47ce-9530-e0f6ff2cae12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:07.892204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed392d46-bf44-47ce-9530-e0f6ff2cae12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:07.894477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed392d46-bf44-47ce-9530-e0f6ff2cae12 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:07.931660 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1272/2534] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:44:07 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:07.935720 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab19f9-f100-4a4d-b609-fc21357dc15f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:07.947913 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab19f9-f100-4a4d-b609-fc21357dc15f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:08.004592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:08.032472 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374a2e4-c458-491d-aa7e-b70a92e77d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:08.033161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374a2e4-c458-491d-aa7e-b70a92e77d7d admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:08.033509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374a2e4-c458-491d-aa7e-b70a92e77d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:08.033902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374a2e4-c458-491d-aa7e-b70a92e77d7d admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:08.035410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374a2e4-c458-491d-aa7e-b70a92e77d7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:08.037766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374a2e4-c458-491d-aa7e-b70a92e77d7d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:08.073167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1263/2535] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:44:07 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:08.077645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed392d46-bf44-47ce-9530-e0f6ff2cae12 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:08.093554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed392d46-bf44-47ce-9530-e0f6ff2cae12 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:08.107164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5510b6-e989-416a-9a96-6d07d2c4e7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:08.107754 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5510b6-e989-416a-9a96-6d07d2c4e7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:08.108036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5510b6-e989-416a-9a96-6d07d2c4e7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:08.108312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5510b6-e989-416a-9a96-6d07d2c4e7ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:08.109013 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5510b6-e989-416a-9a96-6d07d2c4e7ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:08.110760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5510b6-e989-416a-9a96-6d07d2c4e7ea admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:08.135108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1273/2536] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:08.139140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374a2e4-c458-491d-aa7e-b70a92e77d7d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:08.154256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374a2e4-c458-491d-aa7e-b70a92e77d7d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:08.161753 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa5532-2c1a-4621-acee-9006e7fe04ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:08.162654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa5532-2c1a-4621-acee-9006e7fe04ce admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:08.162881 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa5532-2c1a-4621-acee-9006e7fe04ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:08.163247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa5532-2c1a-4621-acee-9006e7fe04ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:08.164196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa5532-2c1a-4621-acee-9006e7fe04ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:08.165845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa5532-2c1a-4621-acee-9006e7fe04ce admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:08.180659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1264/2537] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:44:08.184679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5510b6-e989-416a-9a96-6d07d2c4e7ea admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:08.199832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5510b6-e989-416a-9a96-6d07d2c4e7ea admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:08.209217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660002d7-0218-46fc-9b00-f43d09f2d015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:08.209902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660002d7-0218-46fc-9b00-f43d09f2d015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:08.210186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660002d7-0218-46fc-9b00-f43d09f2d015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:08.211263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660002d7-0218-46fc-9b00-f43d09f2d015 admin admin] PATH_INFO: `/v3/projects/feb6da7425934310bd368e6434513377/users/dc0af6f3b1304e38b8b39c8d6a0ebf24/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:08.239840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660002d7-0218-46fc-9b00-f43d09f2d015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feb6da7425934310bd368e6434513377, user_id=dc0af6f3b1304e38b8b39c8d6a0ebf24, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:08.243830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660002d7-0218-46fc-9b00-f43d09f2d015 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:08.264147 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1274/2538] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:44:08 2022] PUT /identity/v3/projects/feb6da7425934310bd368e6434513377/users/dc0af6f3b1304e38b8b39c8d6a0ebf24/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:44:08.274537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3818596-5989-4da1-8f19-393aa6110b3b None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:08.274824 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3818596-5989-4da1-8f19-393aa6110b3b None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:08.275136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3818596-5989-4da1-8f19-393aa6110b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:08.310441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e3818596-5989-4da1-8f19-393aa6110b3b None None] MFA Rules not processed for user `dc0af6f3b1304e38b8b39c8d6a0ebf24`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:08.311692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e3818596-5989-4da1-8f19-393aa6110b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:08.384676 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1265/2539] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:08.396483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660002d7-0218-46fc-9b00-f43d09f2d015 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:08.398262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-660002d7-0218-46fc-9b00-f43d09f2d015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:08.465201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660002d7-0218-46fc-9b00-f43d09f2d015 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:08.468872 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891033cc-e452-4d82-a759-63b23b4b57d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:08.469490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891033cc-e452-4d82-a759-63b23b4b57d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:08.469773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891033cc-e452-4d82-a759-63b23b4b57d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:08.470049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891033cc-e452-4d82-a759-63b23b4b57d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:08.479253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891033cc-e452-4d82-a759-63b23b4b57d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:08.480744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891033cc-e452-4d82-a759-63b23b4b57d7 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:08.490515 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1275/2540] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:11.112797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3818596-5989-4da1-8f19-393aa6110b3b None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:11.176467 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3818596-5989-4da1-8f19-393aa6110b3b None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:11.180959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:11.181592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:11.181875 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:11.182150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] PATH_INFO: `/v3/users/dc0af6f3b1304e38b8b39c8d6a0ebf24` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:11.183318 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc0af6f3b1304e38b8b39c8d6a0ebf24)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:11.185235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:11.233884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc0af6f3b1304e38b8b39c8d6a0ebf24'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:11.235920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:11.247185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc0af6f3b1304e38b8b39c8d6a0ebf24'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:11.247849 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc0af6f3b1304e38b8b39c8d6a0ebf24'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:11.257508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc0af6f3b1304e38b8b39c8d6a0ebf24'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:11.265422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc0af6f3b1304e38b8b39c8d6a0ebf24'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:11.272077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1266/2541] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:44:11 2022] DELETE /identity/v3/users/dc0af6f3b1304e38b8b39c8d6a0ebf24 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:11.276693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b758509-b628-4005-9bd6-9a7904e1f2b0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:11.277188 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b758509-b628-4005-9bd6-9a7904e1f2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:11.277305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b758509-b628-4005-9bd6-9a7904e1f2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:11.296366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6b758509-b628-4005-9bd6-9a7904e1f2b0 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:11.297554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b758509-b628-4005-9bd6-9a7904e1f2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:11.316543 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:11.361707 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1276/2542] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:11.371417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:11.372839 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:11.430028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6385f-c88d-41c4-9421-339ebdb7c1d6 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:11.433560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d77e0-b409-4b07-94fa-8775c606a44a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:11.434162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d77e0-b409-4b07-94fa-8775c606a44a service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:11.434441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d77e0-b409-4b07-94fa-8775c606a44a service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:11.434713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d77e0-b409-4b07-94fa-8775c606a44a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:11.444289 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d77e0-b409-4b07-94fa-8775c606a44a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:11.445758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d77e0-b409-4b07-94fa-8775c606a44a service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:11.455423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1267/2543] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:11.727807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b758509-b628-4005-9bd6-9a7904e1f2b0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:11.729300 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b758509-b628-4005-9bd6-9a7904e1f2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:11.744154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b758509-b628-4005-9bd6-9a7904e1f2b0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:11.749088 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:11.749767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:11.750093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:11.750600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] PATH_INFO: `/v3/projects/feb6da7425934310bd368e6434513377` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:11.751863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=feb6da7425934310bd368e6434513377)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:11.753992 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:11.801665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feb6da7425934310bd368e6434513377', 'request_id': 'req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:11.808972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feb6da7425934310bd368e6434513377', 'request_id': 'req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:11.814575 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feb6da7425934310bd368e6434513377', 'request_id': 'req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:11.825993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] The token cache is being invalidate because project feb6da7425934310bd368e6434513377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:44:11.826285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:11.826926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:11.828461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1277/2544] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:44:11 2022] DELETE /identity/v3/projects/feb6da7425934310bd368e6434513377 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:13.319160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d77e0-b409-4b07-94fa-8775c606a44a service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:13.320597 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-573d77e0-b409-4b07-94fa-8775c606a44a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:13.333005 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d77e0-b409-4b07-94fa-8775c606a44a service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:13.382468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1306a-fc63-4ec6-a05a-29804be98969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:13.383095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1306a-fc63-4ec6-a05a-29804be98969 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:13.383384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1306a-fc63-4ec6-a05a-29804be98969 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:13.383662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1306a-fc63-4ec6-a05a-29804be98969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:13.385376 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-21f1306a-fc63-4ec6-a05a-29804be98969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:13.447448 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1306a-fc63-4ec6-a05a-29804be98969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:13.448939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1306a-fc63-4ec6-a05a-29804be98969 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:13.459059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1268/2545] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:16.579031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:16.581328 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:16.597181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d07a6-0152-4ab4-ac0e-ad19e7610a6b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:16.602224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311afdbb-6d57-401b-a094-c5281a1525ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:16.602910 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311afdbb-6d57-401b-a094-c5281a1525ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:16.603220 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311afdbb-6d57-401b-a094-c5281a1525ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:16.603500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311afdbb-6d57-401b-a094-c5281a1525ba admin admin] PATH_INFO: `/v3/users/5958b9f3a2c4424db419e7b1c667ca92` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:16.604659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311afdbb-6d57-401b-a094-c5281a1525ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=5958b9f3a2c4424db419e7b1c667ca92)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:16.606588 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311afdbb-6d57-401b-a094-c5281a1525ba admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:16.653083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-311afdbb-6d57-401b-a094-c5281a1525ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5958b9f3a2c4424db419e7b1c667ca92'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:16.660264 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-311afdbb-6d57-401b-a094-c5281a1525ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5958b9f3a2c4424db419e7b1c667ca92'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:16.671977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-311afdbb-6d57-401b-a094-c5281a1525ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5958b9f3a2c4424db419e7b1c667ca92'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:16.672615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-311afdbb-6d57-401b-a094-c5281a1525ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5958b9f3a2c4424db419e7b1c667ca92'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:16.673244 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-311afdbb-6d57-401b-a094-c5281a1525ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5958b9f3a2c4424db419e7b1c667ca92'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:16.686058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1278/2546] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:44:16 2022] DELETE /identity/v3/users/5958b9f3a2c4424db419e7b1c667ca92 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:16.859431 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1306a-fc63-4ec6-a05a-29804be98969 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:16.870915 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1306a-fc63-4ec6-a05a-29804be98969 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:16.889509 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:16.930774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676574a9-ea9a-435a-9f6a-e3245f112832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:16.931444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676574a9-ea9a-435a-9f6a-e3245f112832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:16.931729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676574a9-ea9a-435a-9f6a-e3245f112832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:16.932011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676574a9-ea9a-435a-9f6a-e3245f112832 admin admin] PATH_INFO: `/v3/projects/311edf4e3e534746bed2ff3a2888434d` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:16.933182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676574a9-ea9a-435a-9f6a-e3245f112832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=311edf4e3e534746bed2ff3a2888434d)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:16.935081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676574a9-ea9a-435a-9f6a-e3245f112832 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:16.979737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-676574a9-ea9a-435a-9f6a-e3245f112832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '311edf4e3e534746bed2ff3a2888434d', 'request_id': 'req-676574a9-ea9a-435a-9f6a-e3245f112832'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:16.985131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-676574a9-ea9a-435a-9f6a-e3245f112832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '311edf4e3e534746bed2ff3a2888434d', 'request_id': 'req-676574a9-ea9a-435a-9f6a-e3245f112832'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:16.991956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-676574a9-ea9a-435a-9f6a-e3245f112832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '311edf4e3e534746bed2ff3a2888434d', 'request_id': 'req-676574a9-ea9a-435a-9f6a-e3245f112832'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:17.004046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-676574a9-ea9a-435a-9f6a-e3245f112832 admin admin] The token cache is being invalidate because project 311edf4e3e534746bed2ff3a2888434d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:44:17.004333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-676574a9-ea9a-435a-9f6a-e3245f112832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:17.004940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-676574a9-ea9a-435a-9f6a-e3245f112832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:17.006410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1269/2547] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:44:16 2022] DELETE /identity/v3/projects/311edf4e3e534746bed2ff3a2888434d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:17.011432 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3476a7-7c9d-4581-b5b0-a25815dc1416 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:17.011729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3476a7-7c9d-4581-b5b0-a25815dc1416 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:17.012015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3476a7-7c9d-4581-b5b0-a25815dc1416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:17.031764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3e3476a7-7c9d-4581-b5b0-a25815dc1416 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:17.032986 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e3476a7-7c9d-4581-b5b0-a25815dc1416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:17.046327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:17.093262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1279/2548] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:17.101393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3cb05-2726-405a-98a7-9258fd433764 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:17.101708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3cb05-2726-405a-98a7-9258fd433764 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:17.101989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3cb05-2726-405a-98a7-9258fd433764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:17.122087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7cb3cb05-2726-405a-98a7-9258fd433764 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:17.123320 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7cb3cb05-2726-405a-98a7-9258fd433764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:17.132399 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1270/2549] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:17.136374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3476a7-7c9d-4581-b5b0-a25815dc1416 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:17.146284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3476a7-7c9d-4581-b5b0-a25815dc1416 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:17.150876 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96e260-db29-4074-89d4-3668c4469e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:17.151524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96e260-db29-4074-89d4-3668c4469e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:17.151812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96e260-db29-4074-89d4-3668c4469e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:17.152092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96e260-db29-4074-89d4-3668c4469e1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:17.152786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96e260-db29-4074-89d4-3668c4469e1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:44:17.153097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96e260-db29-4074-89d4-3668c4469e1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:17.154691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96e260-db29-4074-89d4-3668c4469e1d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:17.166079 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1280/2550] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:44:17.170356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3cb05-2726-405a-98a7-9258fd433764 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:17.180142 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3cb05-2726-405a-98a7-9258fd433764 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:17.184784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282cc0e-cc5a-4297-97e8-39866204faff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:17.185319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282cc0e-cc5a-4297-97e8-39866204faff admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:17.185602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282cc0e-cc5a-4297-97e8-39866204faff admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:17.185877 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282cc0e-cc5a-4297-97e8-39866204faff admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:17.186532 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282cc0e-cc5a-4297-97e8-39866204faff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:17.188458 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282cc0e-cc5a-4297-97e8-39866204faff admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:17.215604 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1271/2551] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:44:17 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:17.219889 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96e260-db29-4074-89d4-3668c4469e1d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:17.229740 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96e260-db29-4074-89d4-3668c4469e1d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:17.284895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddb5b6-f53c-4d8d-9013-dc315dd9ceff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:17.285501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddb5b6-f53c-4d8d-9013-dc315dd9ceff admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:17.285852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddb5b6-f53c-4d8d-9013-dc315dd9ceff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:17.286170 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddb5b6-f53c-4d8d-9013-dc315dd9ceff admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:17.286895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ddb5b6-f53c-4d8d-9013-dc315dd9ceff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:17.288830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ddb5b6-f53c-4d8d-9013-dc315dd9ceff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:17.309745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1281/2552] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:44:17 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:17.314019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282cc0e-cc5a-4297-97e8-39866204faff admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:17.325225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282cc0e-cc5a-4297-97e8-39866204faff admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:17.329688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1f572-6d50-4ea6-b42a-d8d689779ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:17.330407 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1f572-6d50-4ea6-b42a-d8d689779ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:17.331029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1f572-6d50-4ea6-b42a-d8d689779ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:17.331347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1f572-6d50-4ea6-b42a-d8d689779ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:17.332068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1f572-6d50-4ea6-b42a-d8d689779ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:17.333661 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1f572-6d50-4ea6-b42a-d8d689779ee4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:17.343494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1272/2553] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:17.347411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddb5b6-f53c-4d8d-9013-dc315dd9ceff admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:17.357283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddb5b6-f53c-4d8d-9013-dc315dd9ceff admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:17.362528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c3811-747f-42d5-83e2-14bffb4f0866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:17.363168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c3811-747f-42d5-83e2-14bffb4f0866 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:17.363461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c3811-747f-42d5-83e2-14bffb4f0866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:17.363746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c3811-747f-42d5-83e2-14bffb4f0866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:17.364437 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c3811-747f-42d5-83e2-14bffb4f0866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:17.366016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c3811-747f-42d5-83e2-14bffb4f0866 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:17.376820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1282/2554] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:17.381616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1f572-6d50-4ea6-b42a-d8d689779ee4 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:17.391750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1f572-6d50-4ea6-b42a-d8d689779ee4 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:17.396870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7479b-a0b9-47cb-aba0-fb55a72f16c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:17.397550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7479b-a0b9-47cb-aba0-fb55a72f16c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:17.397836 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7479b-a0b9-47cb-aba0-fb55a72f16c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:17.398133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7479b-a0b9-47cb-aba0-fb55a72f16c0 admin admin] PATH_INFO: `/v3/projects/49e0013064fc495285dad557b0beef7d/users/1ba46a929ed54c51a33662367289eee5/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:17.413473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7479b-a0b9-47cb-aba0-fb55a72f16c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e0013064fc495285dad557b0beef7d, user_id=1ba46a929ed54c51a33662367289eee5, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:17.416218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7479b-a0b9-47cb-aba0-fb55a72f16c0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:17.429763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1273/2555] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:44:17 2022] PUT /identity/v3/projects/49e0013064fc495285dad557b0beef7d/users/1ba46a929ed54c51a33662367289eee5/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:17.435025 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cca3c-d0a8-4e79-b542-87774b97447f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:17.435315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cca3c-d0a8-4e79-b542-87774b97447f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:17.435595 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cca3c-d0a8-4e79-b542-87774b97447f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:17.454847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c70cca3c-d0a8-4e79-b542-87774b97447f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:17.456335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c70cca3c-d0a8-4e79-b542-87774b97447f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:17.513746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1283/2556] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:17.523518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7479b-a0b9-47cb-aba0-fb55a72f16c0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:17.525239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3dc7479b-a0b9-47cb-aba0-fb55a72f16c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:17.576707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:17.583251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7479b-a0b9-47cb-aba0-fb55a72f16c0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:17.587700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c8946-8c3c-4469-abbb-8122ba7e3300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:17.588308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c8946-8c3c-4469-abbb-8122ba7e3300 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:17.588588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c8946-8c3c-4469-abbb-8122ba7e3300 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:17.588863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c8946-8c3c-4469-abbb-8122ba7e3300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:17.598360 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c8946-8c3c-4469-abbb-8122ba7e3300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:17.599822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c8946-8c3c-4469-abbb-8122ba7e3300 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:17.609048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1274/2557] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:21.817578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e23db-6422-4d94-a45d-7a12b924b5e5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:21.818078 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e23db-6422-4d94-a45d-7a12b924b5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:21.818163 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e23db-6422-4d94-a45d-7a12b924b5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:21.880056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5d1e23db-6422-4d94-a45d-7a12b924b5e5 None None] MFA Rules not processed for user `1ba46a929ed54c51a33662367289eee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:21.881304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5d1e23db-6422-4d94-a45d-7a12b924b5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:21.898480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:21.974466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1284/2558] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:21.989990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c8946-8c3c-4469-abbb-8122ba7e3300 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:21.991555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-530c8946-8c3c-4469-abbb-8122ba7e3300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:22.078022 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c8946-8c3c-4469-abbb-8122ba7e3300 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:22.084587 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d7f45-7427-4692-9ca8-8b238dbb6d8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:22.085244 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d7f45-7427-4692-9ca8-8b238dbb6d8f service nova] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:22.085518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d7f45-7427-4692-9ca8-8b238dbb6d8f service nova] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:22.085797 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d7f45-7427-4692-9ca8-8b238dbb6d8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:22.094517 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d7f45-7427-4692-9ca8-8b238dbb6d8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:22.097411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d7f45-7427-4692-9ca8-8b238dbb6d8f service nova] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:22.110343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1275/2559] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:23.588477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3d6a7-ad4a-4d3e-9197-b34bad28abcc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:23.588991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3d6a7-ad4a-4d3e-9197-b34bad28abcc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:23.589093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3d6a7-ad4a-4d3e-9197-b34bad28abcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:23.608228 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-02c3d6a7-ad4a-4d3e-9197-b34bad28abcc None None] MFA Rules not processed for user `1ba46a929ed54c51a33662367289eee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:23.609542 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-02c3d6a7-ad4a-4d3e-9197-b34bad28abcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:23.616493 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1285/2560] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:44:23.988709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d7f45-7427-4692-9ca8-8b238dbb6d8f service nova] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:23.990181 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4d2d7f45-7427-4692-9ca8-8b238dbb6d8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:24.004093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d7f45-7427-4692-9ca8-8b238dbb6d8f service nova] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:24.007870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f33c6-b6b8-490c-9854-2f7eb905d6cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:24.008914 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f33c6-b6b8-490c-9854-2f7eb905d6cf service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:24.009418 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f33c6-b6b8-490c-9854-2f7eb905d6cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:24.009729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f33c6-b6b8-490c-9854-2f7eb905d6cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:24.011614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-615f33c6-b6b8-490c-9854-2f7eb905d6cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:24.024157 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f33c6-b6b8-490c-9854-2f7eb905d6cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:24.026060 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f33c6-b6b8-490c-9854-2f7eb905d6cf service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:24.036956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1276/2561] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:44:28.042132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3d6a7-ad4a-4d3e-9197-b34bad28abcc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:28.052399 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3d6a7-ad4a-4d3e-9197-b34bad28abcc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:28.055891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10157053-1bde-4879-8512-f45138b0186a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:28.056494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10157053-1bde-4879-8512-f45138b0186a service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:28.056773 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10157053-1bde-4879-8512-f45138b0186a service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:28.057047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10157053-1bde-4879-8512-f45138b0186a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:28.058791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-10157053-1bde-4879-8512-f45138b0186a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:28.117712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10157053-1bde-4879-8512-f45138b0186a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:28.119176 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10157053-1bde-4879-8512-f45138b0186a service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:28.128984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1286/2562] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:33.600493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f33c6-b6b8-490c-9854-2f7eb905d6cf service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:33.609309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f33c6-b6b8-490c-9854-2f7eb905d6cf service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:33.613072 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460c3df-dc4d-4dfc-a6e8-ff946bc01345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:33.613618 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460c3df-dc4d-4dfc-a6e8-ff946bc01345 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:33.613903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460c3df-dc4d-4dfc-a6e8-ff946bc01345 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:33.614182 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460c3df-dc4d-4dfc-a6e8-ff946bc01345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:33.615957 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0460c3df-dc4d-4dfc-a6e8-ff946bc01345 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:33.629500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460c3df-dc4d-4dfc-a6e8-ff946bc01345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:33.630953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460c3df-dc4d-4dfc-a6e8-ff946bc01345 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:33.640613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1277/2563] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:38.124046 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=51 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:40.990170 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10157053-1bde-4879-8512-f45138b0186a service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:40.991764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-10157053-1bde-4879-8512-f45138b0186a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:41.006466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10157053-1bde-4879-8512-f45138b0186a service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:41.011044 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:41.011703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:41.011987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:41.012266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] PATH_INFO: `/v3/users/d714badcdf8e4461ae1f7ebe4f0c82d4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:41.013419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d714badcdf8e4461ae1f7ebe4f0c82d4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:41.015472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:41.063165 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd714badcdf8e4461ae1f7ebe4f0c82d4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.070310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd714badcdf8e4461ae1f7ebe4f0c82d4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.082990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd714badcdf8e4461ae1f7ebe4f0c82d4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.083668 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd714badcdf8e4461ae1f7ebe4f0c82d4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.084279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd714badcdf8e4461ae1f7ebe4f0c82d4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.097194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1287/2564] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:44:40 2022] DELETE /identity/v3/users/d714badcdf8e4461ae1f7ebe4f0c82d4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:41.267340 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460c3df-dc4d-4dfc-a6e8-ff946bc01345 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:41.277325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460c3df-dc4d-4dfc-a6e8-ff946bc01345 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:41.334173 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:41.334731 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:41.335009 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:41.335335 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] PATH_INFO: `/v3/projects/f07c6f16ad7747c5a42b4d0351a44545` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:41.336471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f07c6f16ad7747c5a42b4d0351a44545)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:41.338375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:41.384111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f07c6f16ad7747c5a42b4d0351a44545', 'request_id': 'req-5928c1d0-e332-468b-8ef2-da10d21e4af9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.385262 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:41.390626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f07c6f16ad7747c5a42b4d0351a44545', 'request_id': 'req-5928c1d0-e332-468b-8ef2-da10d21e4af9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.397075 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f07c6f16ad7747c5a42b4d0351a44545', 'request_id': 'req-5928c1d0-e332-468b-8ef2-da10d21e4af9'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.409916 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] The token cache is being invalidate because project f07c6f16ad7747c5a42b4d0351a44545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:44:41.410207 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.410829 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.412347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1278/2565] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:44:41 2022] DELETE /identity/v3/projects/f07c6f16ad7747c5a42b4d0351a44545 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:41.416462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:41.417899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:41.431972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab977c16-e4a0-4ca9-8ea6-32de323c5905 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:41.488752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:41.489387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:41.489673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:41.489955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] PATH_INFO: `/v3/users/9b9897594a6c4aafbb58e94d1caf9b52` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:41.491152 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9897594a6c4aafbb58e94d1caf9b52)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:41.493076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:41.538445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9897594a6c4aafbb58e94d1caf9b52'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.545776 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9897594a6c4aafbb58e94d1caf9b52'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.558283 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9897594a6c4aafbb58e94d1caf9b52'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.558925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9897594a6c4aafbb58e94d1caf9b52'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.559567 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9897594a6c4aafbb58e94d1caf9b52'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.561519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:41.574452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1288/2566] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:44:41 2022] DELETE /identity/v3/users/9b9897594a6c4aafbb58e94d1caf9b52 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:41.749499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:41.751321 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:41.765704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928c1d0-e332-468b-8ef2-da10d21e4af9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:41.832637 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:41.833381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:41.833727 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:41.834087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] PATH_INFO: `/v3/projects/c8c26cad6da4415a97e1816d8da3cfef` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:41.835468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c26cad6da4415a97e1816d8da3cfef)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:41.837790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:41.887782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c26cad6da4415a97e1816d8da3cfef', 'request_id': 'req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.893351 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c26cad6da4415a97e1816d8da3cfef', 'request_id': 'req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.899525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c26cad6da4415a97e1816d8da3cfef', 'request_id': 'req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.913932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] The token cache is being invalidate because project c8c26cad6da4415a97e1816d8da3cfef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:44:41.914229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.914876 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:41.916649 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1279/2567] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:44:41 2022] DELETE /identity/v3/projects/c8c26cad6da4415a97e1816d8da3cfef => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:51.570854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:51.910166 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:53.880608 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:53.882033 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:53.938879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46fd28-7c0a-4d2c-b001-359da43be811 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:53.987826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3d913-90db-4e55-a234-bc9f3dd05cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:53.988436 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3d913-90db-4e55-a234-bc9f3dd05cae service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:53.988718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3d913-90db-4e55-a234-bc9f3dd05cae service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:53.988993 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3d913-90db-4e55-a234-bc9f3dd05cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:53.990726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-daf3d913-90db-4e55-a234-bc9f3dd05cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:54.051902 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3d913-90db-4e55-a234-bc9f3dd05cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:54.053265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3d913-90db-4e55-a234-bc9f3dd05cae service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:54.062724 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1289/2568] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:57.162099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:57.163715 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:57.180144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34bc49-4ce9-44d7-901b-4b062e21fb8f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:57.184821 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3442f-3f15-4174-864d-3ce8abe31f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:57.185462 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3442f-3f15-4174-864d-3ce8abe31f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:57.185746 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3442f-3f15-4174-864d-3ce8abe31f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:57.186081 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3442f-3f15-4174-864d-3ce8abe31f2c admin admin] PATH_INFO: `/v3/users/1e02f414ef2a4990bd1492a63d074319` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:57.187266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3442f-3f15-4174-864d-3ce8abe31f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e02f414ef2a4990bd1492a63d074319)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:57.189207 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3442f-3f15-4174-864d-3ce8abe31f2c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:57.235871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3af3442f-3f15-4174-864d-3ce8abe31f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e02f414ef2a4990bd1492a63d074319'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:57.248254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3af3442f-3f15-4174-864d-3ce8abe31f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e02f414ef2a4990bd1492a63d074319'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:57.248915 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3af3442f-3f15-4174-864d-3ce8abe31f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e02f414ef2a4990bd1492a63d074319'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:57.256596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3af3442f-3f15-4174-864d-3ce8abe31f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e02f414ef2a4990bd1492a63d074319'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:57.263611 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-3af3442f-3f15-4174-864d-3ce8abe31f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e02f414ef2a4990bd1492a63d074319'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:57.269405 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1280/2569] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:44:57 2022] DELETE /identity/v3/users/1e02f414ef2a4990bd1492a63d074319 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:57.444581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3d913-90db-4e55-a234-bc9f3dd05cae service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:57.455494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3d913-90db-4e55-a234-bc9f3dd05cae service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:57.513729 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:57.514366 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:57.514651 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:57.514931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969 admin admin] PATH_INFO: `/v3/projects/4a2b83a4dad243e79144db87357d1897` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:57.516111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2b83a4dad243e79144db87357d1897)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:57.518274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:57.525901 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:57.692580 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2b83a4dad243e79144db87357d1897', 'request_id': 'req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:57.701015 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2b83a4dad243e79144db87357d1897', 'request_id': 'req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:57.706731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2b83a4dad243e79144db87357d1897', 'request_id': 'req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:57.717679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969 admin admin] The token cache is being invalidate because project 4a2b83a4dad243e79144db87357d1897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:44:57.717966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:57.718597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2b1b7ef5-1e02-4c88-b134-a1da82ee4969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:44:57.720121 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1290/2570] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:44:57 2022] DELETE /identity/v3/projects/4a2b83a4dad243e79144db87357d1897 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:57.725748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c428-78af-4a2d-a4c9-656459ad1337 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:57.726034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c428-78af-4a2d-a4c9-656459ad1337 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:57.726314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c428-78af-4a2d-a4c9-656459ad1337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:57.746191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-d421c428-78af-4a2d-a4c9-656459ad1337 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:57.747433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d421c428-78af-4a2d-a4c9-656459ad1337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:57.806764 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1281/2571] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:57.814584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7067526-b002-4497-aef8-3fff67cba86f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:57.814937 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7067526-b002-4497-aef8-3fff67cba86f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:57.815340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7067526-b002-4497-aef8-3fff67cba86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:57.835141 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a7067526-b002-4497-aef8-3fff67cba86f None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:57.837471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7067526-b002-4497-aef8-3fff67cba86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:57.848569 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1291/2572] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:57.852961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c428-78af-4a2d-a4c9-656459ad1337 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:57.863524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c428-78af-4a2d-a4c9-656459ad1337 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:57.868110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f364-ac5e-43da-8b3d-f9c179a1acd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:57.868713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f364-ac5e-43da-8b3d-f9c179a1acd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:57.868996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f364-ac5e-43da-8b3d-f9c179a1acd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:57.869267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f364-ac5e-43da-8b3d-f9c179a1acd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:57.869982 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2f364-ac5e-43da-8b3d-f9c179a1acd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:44:57.870287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2f364-ac5e-43da-8b3d-f9c179a1acd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:57.871940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2f364-ac5e-43da-8b3d-f9c179a1acd4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:57.883867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1282/2573] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:44:57.888333 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7067526-b002-4497-aef8-3fff67cba86f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:57.898705 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7067526-b002-4497-aef8-3fff67cba86f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:57.903506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60cfa2-9301-4ea2-87c9-e9cf3309afc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:57.904232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60cfa2-9301-4ea2-87c9-e9cf3309afc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:57.904518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60cfa2-9301-4ea2-87c9-e9cf3309afc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:57.904794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60cfa2-9301-4ea2-87c9-e9cf3309afc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:57.905457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60cfa2-9301-4ea2-87c9-e9cf3309afc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:57.907379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60cfa2-9301-4ea2-87c9-e9cf3309afc3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:57.935420 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1292/2574] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:44:57 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:44:57.939927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f364-ac5e-43da-8b3d-f9c179a1acd4 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:57.950276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f364-ac5e-43da-8b3d-f9c179a1acd4 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:58.008153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe2eab-636b-4f85-b161-14ab45905906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:58.008769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe2eab-636b-4f85-b161-14ab45905906 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:58.009051 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe2eab-636b-4f85-b161-14ab45905906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:58.009327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe2eab-636b-4f85-b161-14ab45905906 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:58.010047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe2eab-636b-4f85-b161-14ab45905906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:58.012007 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe2eab-636b-4f85-b161-14ab45905906 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:58.020295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:58.036378 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1283/2575] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:44:57 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:58.040275 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60cfa2-9301-4ea2-87c9-e9cf3309afc3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:58.050335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60cfa2-9301-4ea2-87c9-e9cf3309afc3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:58.054824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29830664-698c-424c-89f1-8413acf61845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:58.055460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29830664-698c-424c-89f1-8413acf61845 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:58.055743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29830664-698c-424c-89f1-8413acf61845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:58.056018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29830664-698c-424c-89f1-8413acf61845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:58.056698 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29830664-698c-424c-89f1-8413acf61845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:58.058264 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29830664-698c-424c-89f1-8413acf61845 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:58.068030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1293/2576] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:58.071990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe2eab-636b-4f85-b161-14ab45905906 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:58.082619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe2eab-636b-4f85-b161-14ab45905906 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:58.087252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df48f2-c7e1-4413-8352-c56b09d347c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:58.087853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df48f2-c7e1-4413-8352-c56b09d347c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:58.088139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df48f2-c7e1-4413-8352-c56b09d347c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:58.088420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df48f2-c7e1-4413-8352-c56b09d347c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:58.089177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df48f2-c7e1-4413-8352-c56b09d347c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:58.090782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df48f2-c7e1-4413-8352-c56b09d347c4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:58.102256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1284/2577] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:58.106353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29830664-698c-424c-89f1-8413acf61845 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:58.116279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29830664-698c-424c-89f1-8413acf61845 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:58.120734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc145b-3f92-4bd2-b7a6-38dba3aec8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:58.121379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc145b-3f92-4bd2-b7a6-38dba3aec8c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:58.121657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc145b-3f92-4bd2-b7a6-38dba3aec8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:58.121934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc145b-3f92-4bd2-b7a6-38dba3aec8c0 admin admin] PATH_INFO: `/v3/projects/e18bce7f2f9248b7a2ff46bb60ccc6a9/users/855c9ac4f4a04818b63dd7a5e8f5419c/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:58.136868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc145b-3f92-4bd2-b7a6-38dba3aec8c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18bce7f2f9248b7a2ff46bb60ccc6a9, user_id=855c9ac4f4a04818b63dd7a5e8f5419c, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:58.139492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc145b-3f92-4bd2-b7a6-38dba3aec8c0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:58.152368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1294/2578] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:44:58 2022] PUT /identity/v3/projects/e18bce7f2f9248b7a2ff46bb60ccc6a9/users/855c9ac4f4a04818b63dd7a5e8f5419c/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:44:58.157606 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba239e-c7da-4546-9d9a-0afa768f4dae None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:58.157961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba239e-c7da-4546-9d9a-0afa768f4dae None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:58.159573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba239e-c7da-4546-9d9a-0afa768f4dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:58.178778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-caba239e-c7da-4546-9d9a-0afa768f4dae None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:58.180007 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-caba239e-c7da-4546-9d9a-0afa768f4dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:58.238679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1285/2579] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:58.248491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc145b-3f92-4bd2-b7a6-38dba3aec8c0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:58.250439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0cc145b-3f92-4bd2-b7a6-38dba3aec8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:58.306879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc145b-3f92-4bd2-b7a6-38dba3aec8c0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:58.310478 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febe393-d247-4bfb-870e-1c9afe6a641f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:58.311099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febe393-d247-4bfb-870e-1c9afe6a641f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:58.311389 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febe393-d247-4bfb-870e-1c9afe6a641f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:58.311664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febe393-d247-4bfb-870e-1c9afe6a641f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:58.321445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febe393-d247-4bfb-870e-1c9afe6a641f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:58.322852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febe393-d247-4bfb-870e-1c9afe6a641f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:58.332798 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1295/2580] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:44:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:44:58.938793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba99753-03fe-4e55-ab41-ba3584ea8d4c None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:58.939113 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba99753-03fe-4e55-ab41-ba3584ea8d4c None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:58.940018 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba99753-03fe-4e55-ab41-ba3584ea8d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:58.972526 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7ba99753-03fe-4e55-ab41-ba3584ea8d4c None None] MFA Rules not processed for user `855c9ac4f4a04818b63dd7a5e8f5419c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:44:58.973937 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7ba99753-03fe-4e55-ab41-ba3584ea8d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:44:59.045355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1286/2581] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:44:58 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:44:59.050492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febe393-d247-4bfb-870e-1c9afe6a641f service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:59.060253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febe393-d247-4bfb-870e-1c9afe6a641f service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:59.065108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57611c06-f2c4-4c2a-9e2f-f6b710ca74b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:59.065483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57611c06-f2c4-4c2a-9e2f-f6b710ca74b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:59.065763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57611c06-f2c4-4c2a-9e2f-f6b710ca74b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:59.066038 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57611c06-f2c4-4c2a-9e2f-f6b710ca74b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:59.066694 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57611c06-f2c4-4c2a-9e2f-f6b710ca74b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:59.068671 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57611c06-f2c4-4c2a-9e2f-f6b710ca74b5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:59.101659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1296/2582] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:44:59 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:44:59.106006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba99753-03fe-4e55-ab41-ba3584ea8d4c None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:59.120174 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba99753-03fe-4e55-ab41-ba3584ea8d4c None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:59.176669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e1039-57b1-4f27-9a8f-5a795db425ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:59.177312 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e1039-57b1-4f27-9a8f-5a795db425ab admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:59.177593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e1039-57b1-4f27-9a8f-5a795db425ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:59.177867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e1039-57b1-4f27-9a8f-5a795db425ab admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:59.178527 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e1039-57b1-4f27-9a8f-5a795db425ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:59.180506 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e1039-57b1-4f27-9a8f-5a795db425ab admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:59.196116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:59.203254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1287/2583] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:44:59 2022] POST /identity/v3/users => generated 309 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:44:59.207049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57611c06-f2c4-4c2a-9e2f-f6b710ca74b5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:59.217179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57611c06-f2c4-4c2a-9e2f-f6b710ca74b5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:59.222491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3adcc-3183-42bf-a081-ff3a364ef6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:59.223128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3adcc-3183-42bf-a081-ff3a364ef6ec admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:59.223416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3adcc-3183-42bf-a081-ff3a364ef6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:59.223711 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3adcc-3183-42bf-a081-ff3a364ef6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:59.224373 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3adcc-3183-42bf-a081-ff3a364ef6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:59.225945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3adcc-3183-42bf-a081-ff3a364ef6ec admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:59.235858 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1297/2584] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:44:59.240747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e1039-57b1-4f27-9a8f-5a795db425ab admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:59.250740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e1039-57b1-4f27-9a8f-5a795db425ab admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:59.255427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8e9ce-4954-41e8-a14c-d078721b6c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:59.256077 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8e9ce-4954-41e8-a14c-d078721b6c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:59.256359 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8e9ce-4954-41e8-a14c-d078721b6c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:59.256633 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8e9ce-4954-41e8-a14c-d078721b6c94 admin admin] PATH_INFO: `/v3/projects/ff17d406fe33450f9249932bf6fa21a3/users/1ab6a4cf4e5349dda4c6b4945e43a2ed/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:59.271240 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8e9ce-4954-41e8-a14c-d078721b6c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff17d406fe33450f9249932bf6fa21a3, user_id=1ab6a4cf4e5349dda4c6b4945e43a2ed, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:59.273874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8e9ce-4954-41e8-a14c-d078721b6c94 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:59.287406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1288/2585] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:44:59 2022] PUT /identity/v3/projects/ff17d406fe33450f9249932bf6fa21a3/users/1ab6a4cf4e5349dda4c6b4945e43a2ed/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:44:59.291412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3adcc-3183-42bf-a081-ff3a364ef6ec admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:59.296512 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:44:59.303453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3adcc-3183-42bf-a081-ff3a364ef6ec admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:59.359486 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff79d9a-c1c1-4406-953b-fe07d9658bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:59.360026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff79d9a-c1c1-4406-953b-fe07d9658bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:59.360309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff79d9a-c1c1-4406-953b-fe07d9658bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:59.360586 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff79d9a-c1c1-4406-953b-fe07d9658bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:59.361834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff79d9a-c1c1-4406-953b-fe07d9658bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:59.363453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff79d9a-c1c1-4406-953b-fe07d9658bb3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:59.373337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1298/2586] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:44:59.378224 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8e9ce-4954-41e8-a14c-d078721b6c94 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:44:59.389050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8e9ce-4954-41e8-a14c-d078721b6c94 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:44:59.394403 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c494c-42f4-4bb9-8c33-fbb2927aa2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:44:59.395093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c494c-42f4-4bb9-8c33-fbb2927aa2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:44:59.395379 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c494c-42f4-4bb9-8c33-fbb2927aa2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:44:59.395693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c494c-42f4-4bb9-8c33-fbb2927aa2f2 admin admin] PATH_INFO: `/v3/projects/ff17d406fe33450f9249932bf6fa21a3/users/1ab6a4cf4e5349dda4c6b4945e43a2ed/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:44:59.397644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c494c-42f4-4bb9-8c33-fbb2927aa2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff17d406fe33450f9249932bf6fa21a3, user_id=1ab6a4cf4e5349dda4c6b4945e43a2ed, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:44:59.400350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c494c-42f4-4bb9-8c33-fbb2927aa2f2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:44:59.411403 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1289/2587] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:44:59 2022] PUT /identity/v3/projects/ff17d406fe33450f9249932bf6fa21a3/users/1ab6a4cf4e5349dda4c6b4945e43a2ed/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:00.042680 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af89c5-1c0a-4f4b-b04c-33a20517e953 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:00.043049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af89c5-1c0a-4f4b-b04c-33a20517e953 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:00.043425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af89c5-1c0a-4f4b-b04c-33a20517e953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:00.071295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e1af89c5-1c0a-4f4b-b04c-33a20517e953 None None] MFA Rules not processed for user `1ab6a4cf4e5349dda4c6b4945e43a2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:00.072606 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e1af89c5-1c0a-4f4b-b04c-33a20517e953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:00.144656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1299/2588] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:00 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:45:01.368761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c494c-42f4-4bb9-8c33-fbb2927aa2f2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:01.432656 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c494c-42f4-4bb9-8c33-fbb2927aa2f2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:01.437257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:01.437882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:01.438167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:01.438448 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] PATH_INFO: `/v3/users/855c9ac4f4a04818b63dd7a5e8f5419c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:01.439662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] RBAC: Authorizing `identity:delete_user(user_id=855c9ac4f4a04818b63dd7a5e8f5419c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:01.441575 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:01.486970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '855c9ac4f4a04818b63dd7a5e8f5419c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:01.498763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '855c9ac4f4a04818b63dd7a5e8f5419c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:01.499709 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '855c9ac4f4a04818b63dd7a5e8f5419c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:01.506447 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '855c9ac4f4a04818b63dd7a5e8f5419c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:01.514419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '855c9ac4f4a04818b63dd7a5e8f5419c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:01.520445 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1290/2589] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:01 2022] DELETE /identity/v3/users/855c9ac4f4a04818b63dd7a5e8f5419c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:01.697660 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af89c5-1c0a-4f4b-b04c-33a20517e953 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:01.710774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af89c5-1c0a-4f4b-b04c-33a20517e953 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:01.774683 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:01.775352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:01.775637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:01.775913 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] PATH_INFO: `/v3/projects/e18bce7f2f9248b7a2ff46bb60ccc6a9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:01.777049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18bce7f2f9248b7a2ff46bb60ccc6a9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:01.778923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:01.810421 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:01.828159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18bce7f2f9248b7a2ff46bb60ccc6a9', 'request_id': 'req-917de224-f5b9-4bfe-aab2-67904fcd27a6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:01.835662 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18bce7f2f9248b7a2ff46bb60ccc6a9', 'request_id': 'req-917de224-f5b9-4bfe-aab2-67904fcd27a6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:01.841318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18bce7f2f9248b7a2ff46bb60ccc6a9', 'request_id': 'req-917de224-f5b9-4bfe-aab2-67904fcd27a6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:01.852744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] The token cache is being invalidate because project e18bce7f2f9248b7a2ff46bb60ccc6a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:01.853035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:01.853670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:01.855180 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1300/2590] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:01 2022] DELETE /identity/v3/projects/e18bce7f2f9248b7a2ff46bb60ccc6a9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:01.859189 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:01.860578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:01.874578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ea43c-6f3c-4420-b81c-06e26c74a39c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:01.912952 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:01.933396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cd610-5a4d-4c49-b2f7-9b4fda1ea348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:01.934037 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cd610-5a4d-4c49-b2f7-9b4fda1ea348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:01.934322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cd610-5a4d-4c49-b2f7-9b4fda1ea348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:01.934602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cd610-5a4d-4c49-b2f7-9b4fda1ea348 admin admin] PATH_INFO: `/v3/users/1ab6a4cf4e5349dda4c6b4945e43a2ed` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:01.935859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cd610-5a4d-4c49-b2f7-9b4fda1ea348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ab6a4cf4e5349dda4c6b4945e43a2ed)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:01.937749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cd610-5a4d-4c49-b2f7-9b4fda1ea348 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:01.984424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-aa4cd610-5a4d-4c49-b2f7-9b4fda1ea348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ab6a4cf4e5349dda4c6b4945e43a2ed'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:01.997018 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-aa4cd610-5a4d-4c49-b2f7-9b4fda1ea348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ab6a4cf4e5349dda4c6b4945e43a2ed'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:01.997702 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-aa4cd610-5a4d-4c49-b2f7-9b4fda1ea348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ab6a4cf4e5349dda4c6b4945e43a2ed'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:02.005383 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-aa4cd610-5a4d-4c49-b2f7-9b4fda1ea348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ab6a4cf4e5349dda4c6b4945e43a2ed'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:02.012589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-aa4cd610-5a4d-4c49-b2f7-9b4fda1ea348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ab6a4cf4e5349dda4c6b4945e43a2ed'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:02.018614 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1291/2591] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:01 2022] DELETE /identity/v3/users/1ab6a4cf4e5349dda4c6b4945e43a2ed => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:02.196391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:02.197924 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:02.210383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917de224-f5b9-4bfe-aab2-67904fcd27a6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:02.267005 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d639e10-e3f2-443c-b59d-06ef3eb2be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:02.267804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d639e10-e3f2-443c-b59d-06ef3eb2be29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:02.268089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d639e10-e3f2-443c-b59d-06ef3eb2be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:02.268370 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d639e10-e3f2-443c-b59d-06ef3eb2be29 admin admin] PATH_INFO: `/v3/projects/ff17d406fe33450f9249932bf6fa21a3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:02.269551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d639e10-e3f2-443c-b59d-06ef3eb2be29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff17d406fe33450f9249932bf6fa21a3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:02.271498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d639e10-e3f2-443c-b59d-06ef3eb2be29 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:02.317216 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d639e10-e3f2-443c-b59d-06ef3eb2be29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff17d406fe33450f9249932bf6fa21a3', 'request_id': 'req-9d639e10-e3f2-443c-b59d-06ef3eb2be29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:02.324636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d639e10-e3f2-443c-b59d-06ef3eb2be29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff17d406fe33450f9249932bf6fa21a3', 'request_id': 'req-9d639e10-e3f2-443c-b59d-06ef3eb2be29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:02.330707 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d639e10-e3f2-443c-b59d-06ef3eb2be29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff17d406fe33450f9249932bf6fa21a3', 'request_id': 'req-9d639e10-e3f2-443c-b59d-06ef3eb2be29'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:02.341691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d639e10-e3f2-443c-b59d-06ef3eb2be29 admin admin] The token cache is being invalidate because project ff17d406fe33450f9249932bf6fa21a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:02.342078 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d639e10-e3f2-443c-b59d-06ef3eb2be29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:02.342713 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d639e10-e3f2-443c-b59d-06ef3eb2be29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:02.344253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1301/2592] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:02 2022] DELETE /identity/v3/projects/ff17d406fe33450f9249932bf6fa21a3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:02.349899 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd8ce4-c169-4814-9bd2-5af00d10f166 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:02.350244 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd8ce4-c169-4814-9bd2-5af00d10f166 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:02.350530 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd8ce4-c169-4814-9bd2-5af00d10f166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:02.370331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-13fd8ce4-c169-4814-9bd2-5af00d10f166 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:02.371590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-13fd8ce4-c169-4814-9bd2-5af00d10f166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:02.432489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1292/2593] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:02.440563 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c9336-bf83-446f-999d-f0cd2cbc43db None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:02.440983 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c9336-bf83-446f-999d-f0cd2cbc43db None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:02.441262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c9336-bf83-446f-999d-f0cd2cbc43db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:02.461558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-863c9336-bf83-446f-999d-f0cd2cbc43db None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:02.462763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-863c9336-bf83-446f-999d-f0cd2cbc43db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:02.471648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1302/2594] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:02.476015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd8ce4-c169-4814-9bd2-5af00d10f166 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:02.485679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd8ce4-c169-4814-9bd2-5af00d10f166 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:02.490689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2367c-2b7e-40c4-a50a-f40b9b378d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:02.491342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2367c-2b7e-40c4-a50a-f40b9b378d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:02.491619 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2367c-2b7e-40c4-a50a-f40b9b378d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:02.491900 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2367c-2b7e-40c4-a50a-f40b9b378d0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:02.492594 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2367c-2b7e-40c4-a50a-f40b9b378d0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:02.492909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2367c-2b7e-40c4-a50a-f40b9b378d0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:02.494507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2367c-2b7e-40c4-a50a-f40b9b378d0f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:02.506459 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1293/2595] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:02.510742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c9336-bf83-446f-999d-f0cd2cbc43db None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:02.521088 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c9336-bf83-446f-999d-f0cd2cbc43db None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:02.525759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ef027-f5e6-4dd0-b465-fa1b2457101c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:02.526361 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ef027-f5e6-4dd0-b465-fa1b2457101c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:02.526645 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ef027-f5e6-4dd0-b465-fa1b2457101c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:02.526920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ef027-f5e6-4dd0-b465-fa1b2457101c admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:02.527657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ef027-f5e6-4dd0-b465-fa1b2457101c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:02.529535 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ef027-f5e6-4dd0-b465-fa1b2457101c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:02.556793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1303/2596] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:45:02 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:02.561006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2367c-2b7e-40c4-a50a-f40b9b378d0f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:02.571200 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2367c-2b7e-40c4-a50a-f40b9b378d0f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:02.615175 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:02.629593 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963d19c-9c50-4e17-91ba-8b909a768c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:02.630217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963d19c-9c50-4e17-91ba-8b909a768c47 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:02.630498 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963d19c-9c50-4e17-91ba-8b909a768c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:02.630769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963d19c-9c50-4e17-91ba-8b909a768c47 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:02.631470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963d19c-9c50-4e17-91ba-8b909a768c47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:02.633381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963d19c-9c50-4e17-91ba-8b909a768c47 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:02.653929 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1294/2597] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:45:02 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:02.658795 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ef027-f5e6-4dd0-b465-fa1b2457101c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:02.668777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ef027-f5e6-4dd0-b465-fa1b2457101c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:02.673282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0af01-f6b8-42d6-9757-b54ab52bc96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:02.673982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0af01-f6b8-42d6-9757-b54ab52bc96e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:02.674433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0af01-f6b8-42d6-9757-b54ab52bc96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:02.674708 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0af01-f6b8-42d6-9757-b54ab52bc96e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:02.675441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0af01-f6b8-42d6-9757-b54ab52bc96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:02.677018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0af01-f6b8-42d6-9757-b54ab52bc96e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:02.686663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1304/2598] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:45:02.690889 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963d19c-9c50-4e17-91ba-8b909a768c47 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:02.700686 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963d19c-9c50-4e17-91ba-8b909a768c47 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:02.706291 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6d3b1-38dd-4d49-b642-4c906318b7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:02.706895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6d3b1-38dd-4d49-b642-4c906318b7e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:02.707216 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6d3b1-38dd-4d49-b642-4c906318b7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:02.707498 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6d3b1-38dd-4d49-b642-4c906318b7e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:02.708192 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6d3b1-38dd-4d49-b642-4c906318b7e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:02.709769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6d3b1-38dd-4d49-b642-4c906318b7e0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:02.721572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1295/2599] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:02.725702 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0af01-f6b8-42d6-9757-b54ab52bc96e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:02.731998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:02.735702 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0af01-f6b8-42d6-9757-b54ab52bc96e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:02.742247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de2695-fd9c-4df4-a1ce-016e7bf78c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:02.743184 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de2695-fd9c-4df4-a1ce-016e7bf78c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:02.743537 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de2695-fd9c-4df4-a1ce-016e7bf78c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:02.743831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de2695-fd9c-4df4-a1ce-016e7bf78c75 admin admin] PATH_INFO: `/v3/projects/55e3381c63054ed39a8db7de855f7a0d/users/b051bd83eede41b7b0f45e7c8c6b7423/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:02.759620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de2695-fd9c-4df4-a1ce-016e7bf78c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e3381c63054ed39a8db7de855f7a0d, user_id=b051bd83eede41b7b0f45e7c8c6b7423, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:02.762252 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de2695-fd9c-4df4-a1ce-016e7bf78c75 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:02.776574 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1305/2600] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:02 2022] PUT /identity/v3/projects/55e3381c63054ed39a8db7de855f7a0d/users/b051bd83eede41b7b0f45e7c8c6b7423/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:02.787643 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d3df8-0da7-409d-aea1-c310c64afa82 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:02.787934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d3df8-0da7-409d-aea1-c310c64afa82 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:02.788212 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d3df8-0da7-409d-aea1-c310c64afa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:02.816235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-db2d3df8-0da7-409d-aea1-c310c64afa82 None None] MFA Rules not processed for user `b051bd83eede41b7b0f45e7c8c6b7423`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:02.817444 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-db2d3df8-0da7-409d-aea1-c310c64afa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:02.879514 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1296/2601] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:45:02.884111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de2695-fd9c-4df4-a1ce-016e7bf78c75 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:02.944558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de2695-fd9c-4df4-a1ce-016e7bf78c75 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:02.949305 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67c481-7256-4b0a-964c-abb3bb7f0406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:02.949932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67c481-7256-4b0a-964c-abb3bb7f0406 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:02.950218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67c481-7256-4b0a-964c-abb3bb7f0406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:02.950498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67c481-7256-4b0a-964c-abb3bb7f0406 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:02.951210 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67c481-7256-4b0a-964c-abb3bb7f0406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:02.953111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67c481-7256-4b0a-964c-abb3bb7f0406 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:02.979797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1306/2602] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:45:02 2022] POST /identity/v3/projects => generated 389 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:45:02.983813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d3df8-0da7-409d-aea1-c310c64afa82 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:02.994350 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d3df8-0da7-409d-aea1-c310c64afa82 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:03.049296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322522d1-bcb8-4a6a-9e99-a7b73d3cfe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:03.049762 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322522d1-bcb8-4a6a-9e99-a7b73d3cfe16 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:03.050057 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322522d1-bcb8-4a6a-9e99-a7b73d3cfe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:03.050309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322522d1-bcb8-4a6a-9e99-a7b73d3cfe16 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:03.050956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322522d1-bcb8-4a6a-9e99-a7b73d3cfe16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:03.052901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322522d1-bcb8-4a6a-9e99-a7b73d3cfe16 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:03.073231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1297/2603] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:45:02 2022] POST /identity/v3/users => generated 301 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:03.077172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67c481-7256-4b0a-964c-abb3bb7f0406 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:03.087932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67c481-7256-4b0a-964c-abb3bb7f0406 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:03.092464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27586d90-fa9e-4b96-b38a-eaad312a2518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:03.093062 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27586d90-fa9e-4b96-b38a-eaad312a2518 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:03.093360 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27586d90-fa9e-4b96-b38a-eaad312a2518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:03.093621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27586d90-fa9e-4b96-b38a-eaad312a2518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:03.094306 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27586d90-fa9e-4b96-b38a-eaad312a2518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:03.096036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27586d90-fa9e-4b96-b38a-eaad312a2518 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:03.106184 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1307/2604] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:03.110138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322522d1-bcb8-4a6a-9e99-a7b73d3cfe16 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:03.120064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322522d1-bcb8-4a6a-9e99-a7b73d3cfe16 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:03.124829 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46f3c6-0716-4ea7-950b-fee759c79147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:03.125469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46f3c6-0716-4ea7-950b-fee759c79147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:03.125749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46f3c6-0716-4ea7-950b-fee759c79147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:03.126023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46f3c6-0716-4ea7-950b-fee759c79147 admin admin] PATH_INFO: `/v3/projects/81ac6f89bbc74d15bd2be5a486b62548/users/e335ac8583894bdd965b4b7a832a91ff/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:03.142158 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46f3c6-0716-4ea7-950b-fee759c79147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ac6f89bbc74d15bd2be5a486b62548, user_id=e335ac8583894bdd965b4b7a832a91ff, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:03.145443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46f3c6-0716-4ea7-950b-fee759c79147 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:03.157272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1298/2605] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:03 2022] PUT /identity/v3/projects/81ac6f89bbc74d15bd2be5a486b62548/users/e335ac8583894bdd965b4b7a832a91ff/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:03.161416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27586d90-fa9e-4b96-b38a-eaad312a2518 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:03.171402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27586d90-fa9e-4b96-b38a-eaad312a2518 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:03.226325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc74bd-e7ec-4aea-bb01-cefd6ec8052c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:03.226867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc74bd-e7ec-4aea-bb01-cefd6ec8052c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:03.227187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc74bd-e7ec-4aea-bb01-cefd6ec8052c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:03.227492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc74bd-e7ec-4aea-bb01-cefd6ec8052c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:03.228160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc74bd-e7ec-4aea-bb01-cefd6ec8052c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:03.229741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc74bd-e7ec-4aea-bb01-cefd6ec8052c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:03.239763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1308/2606] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:03.243973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46f3c6-0716-4ea7-950b-fee759c79147 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:03.255084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46f3c6-0716-4ea7-950b-fee759c79147 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:03.259569 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd703d-23db-4527-a180-49df4c04969b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:03.260210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd703d-23db-4527-a180-49df4c04969b admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:03.260488 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd703d-23db-4527-a180-49df4c04969b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:03.260765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd703d-23db-4527-a180-49df4c04969b admin admin] PATH_INFO: `/v3/projects/81ac6f89bbc74d15bd2be5a486b62548/users/e335ac8583894bdd965b4b7a832a91ff/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:03.262691 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd703d-23db-4527-a180-49df4c04969b admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ac6f89bbc74d15bd2be5a486b62548, user_id=e335ac8583894bdd965b4b7a832a91ff, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:03.265322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd703d-23db-4527-a180-49df4c04969b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:03.275514 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1299/2607] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:03 2022] PUT /identity/v3/projects/81ac6f89bbc74d15bd2be5a486b62548/users/e335ac8583894bdd965b4b7a832a91ff/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:03.286734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559fdb2-4aaa-48f5-a43c-35f56e527c72 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:03.287031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559fdb2-4aaa-48f5-a43c-35f56e527c72 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:03.287349 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559fdb2-4aaa-48f5-a43c-35f56e527c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:03.315461 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9559fdb2-4aaa-48f5-a43c-35f56e527c72 None None] MFA Rules not processed for user `e335ac8583894bdd965b4b7a832a91ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:03.316681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9559fdb2-4aaa-48f5-a43c-35f56e527c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:03.355236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:03.385353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1309/2608] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:03.395933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd703d-23db-4527-a180-49df4c04969b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:03.397626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-d2cd703d-23db-4527-a180-49df4c04969b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:03.407249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:03.455087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd703d-23db-4527-a180-49df4c04969b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:03.458665 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faab398-6262-4b4f-ba92-2ee115f24753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:03.459314 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faab398-6262-4b4f-ba92-2ee115f24753 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:03.459595 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faab398-6262-4b4f-ba92-2ee115f24753 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:03.459868 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faab398-6262-4b4f-ba92-2ee115f24753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:03.468611 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faab398-6262-4b4f-ba92-2ee115f24753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:03.470024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faab398-6262-4b4f-ba92-2ee115f24753 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:03.479404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1300/2609] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:03.616341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559fdb2-4aaa-48f5-a43c-35f56e527c72 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:03.676128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559fdb2-4aaa-48f5-a43c-35f56e527c72 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:03.680500 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:03.681120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:03.681401 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:03.681676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] PATH_INFO: `/v3/users/b051bd83eede41b7b0f45e7c8c6b7423` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:03.682796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b051bd83eede41b7b0f45e7c8c6b7423)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:03.684894 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:03.730710 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b051bd83eede41b7b0f45e7c8c6b7423'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:03.738572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b051bd83eede41b7b0f45e7c8c6b7423'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:03.751243 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b051bd83eede41b7b0f45e7c8c6b7423'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:03.751935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b051bd83eede41b7b0f45e7c8c6b7423'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:03.752549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b051bd83eede41b7b0f45e7c8c6b7423'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:03.764491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1310/2610] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:03 2022] DELETE /identity/v3/users/b051bd83eede41b7b0f45e7c8c6b7423 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:03.769313 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf725c-a4a9-49c9-ab48-ac56bcce48ac None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:03.769895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf725c-a4a9-49c9-ab48-ac56bcce48ac None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:03.770239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf725c-a4a9-49c9-ab48-ac56bcce48ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:03.789016 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-46cf725c-a4a9-49c9-ab48-ac56bcce48ac None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:03.790208 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-46cf725c-a4a9-49c9-ab48-ac56bcce48ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:03.846785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1301/2611] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:03.856678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:03.858176 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:03.915332 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af516a07-f1ee-4dfd-9937-d7aa143b2723 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:03.918928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c4851-9290-4725-9285-0fcbefde6fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:03.919560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c4851-9290-4725-9285-0fcbefde6fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:03.919840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c4851-9290-4725-9285-0fcbefde6fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:03.920113 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c4851-9290-4725-9285-0fcbefde6fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:03.928907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c4851-9290-4725-9285-0fcbefde6fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:03.930312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c4851-9290-4725-9285-0fcbefde6fb1 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:03.934192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:03.941557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1311/2612] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:04.205884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf725c-a4a9-49c9-ab48-ac56bcce48ac None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:04.207331 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-46cf725c-a4a9-49c9-ab48-ac56bcce48ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:04.219468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf725c-a4a9-49c9-ab48-ac56bcce48ac None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:04.224879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:04.225525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:04.225940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:04.226225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] PATH_INFO: `/v3/projects/55e3381c63054ed39a8db7de855f7a0d` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:04.227451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e3381c63054ed39a8db7de855f7a0d)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:04.229366 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:04.277545 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e3381c63054ed39a8db7de855f7a0d', 'request_id': 'req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.283165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e3381c63054ed39a8db7de855f7a0d', 'request_id': 'req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.289638 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e3381c63054ed39a8db7de855f7a0d', 'request_id': 'req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.303668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] The token cache is being invalidate because project 55e3381c63054ed39a8db7de855f7a0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:04.303959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.304581 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.306068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1302/2613] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:04 2022] DELETE /identity/v3/projects/55e3381c63054ed39a8db7de855f7a0d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:04.310076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c4851-9290-4725-9285-0fcbefde6fb1 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:04.320182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c4851-9290-4725-9285-0fcbefde6fb1 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:04.376006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e129c7-3f89-440d-8d33-6f6e2a289d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:04.377130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e129c7-3f89-440d-8d33-6f6e2a289d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:04.377408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e129c7-3f89-440d-8d33-6f6e2a289d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:04.377678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e129c7-3f89-440d-8d33-6f6e2a289d06 admin admin] PATH_INFO: `/v3/users/e335ac8583894bdd965b4b7a832a91ff` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:04.378818 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e129c7-3f89-440d-8d33-6f6e2a289d06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e335ac8583894bdd965b4b7a832a91ff)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:04.380743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e129c7-3f89-440d-8d33-6f6e2a289d06 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:04.424813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-26e129c7-3f89-440d-8d33-6f6e2a289d06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e335ac8583894bdd965b4b7a832a91ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.431801 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-26e129c7-3f89-440d-8d33-6f6e2a289d06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e335ac8583894bdd965b4b7a832a91ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.444648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-26e129c7-3f89-440d-8d33-6f6e2a289d06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e335ac8583894bdd965b4b7a832a91ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.445279 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-26e129c7-3f89-440d-8d33-6f6e2a289d06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e335ac8583894bdd965b4b7a832a91ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.445879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-26e129c7-3f89-440d-8d33-6f6e2a289d06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e335ac8583894bdd965b4b7a832a91ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.458725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1312/2614] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:04 2022] DELETE /identity/v3/users/e335ac8583894bdd965b4b7a832a91ff => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:04.730722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:04.732281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:04.747648 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb980f-2b1c-4ae4-8921-48d33cccd8bc admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:04.787588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:04.816178 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84365f2c-7e9e-428e-8e81-8ba92fda480b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:04.816740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84365f2c-7e9e-428e-8e81-8ba92fda480b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:04.817025 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84365f2c-7e9e-428e-8e81-8ba92fda480b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:04.817301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84365f2c-7e9e-428e-8e81-8ba92fda480b admin admin] PATH_INFO: `/v3/projects/81ac6f89bbc74d15bd2be5a486b62548` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:04.818446 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84365f2c-7e9e-428e-8e81-8ba92fda480b admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ac6f89bbc74d15bd2be5a486b62548)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:04.820392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84365f2c-7e9e-428e-8e81-8ba92fda480b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:04.865102 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-84365f2c-7e9e-428e-8e81-8ba92fda480b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ac6f89bbc74d15bd2be5a486b62548', 'request_id': 'req-84365f2c-7e9e-428e-8e81-8ba92fda480b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.870526 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-84365f2c-7e9e-428e-8e81-8ba92fda480b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ac6f89bbc74d15bd2be5a486b62548', 'request_id': 'req-84365f2c-7e9e-428e-8e81-8ba92fda480b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.877490 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-84365f2c-7e9e-428e-8e81-8ba92fda480b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ac6f89bbc74d15bd2be5a486b62548', 'request_id': 'req-84365f2c-7e9e-428e-8e81-8ba92fda480b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.889685 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-84365f2c-7e9e-428e-8e81-8ba92fda480b admin admin] The token cache is being invalidate because project 81ac6f89bbc74d15bd2be5a486b62548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:04.889976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-84365f2c-7e9e-428e-8e81-8ba92fda480b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.890587 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-84365f2c-7e9e-428e-8e81-8ba92fda480b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:04.892184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1303/2615] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:04 2022] DELETE /identity/v3/projects/81ac6f89bbc74d15bd2be5a486b62548 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:04.896950 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720376b1-974d-4888-ac74-6ec7aee90d86 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:04.897448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720376b1-974d-4888-ac74-6ec7aee90d86 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:04.897585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720376b1-974d-4888-ac74-6ec7aee90d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:04.916393 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-720376b1-974d-4888-ac74-6ec7aee90d86 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:04.917581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-720376b1-974d-4888-ac74-6ec7aee90d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:04.976946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1313/2616] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:04.984807 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de2e40-523f-4527-819d-660a1738ed8a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:04.985098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de2e40-523f-4527-819d-660a1738ed8a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:04.985382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de2e40-523f-4527-819d-660a1738ed8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.006073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-22de2e40-523f-4527-819d-660a1738ed8a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:05.007304 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-22de2e40-523f-4527-819d-660a1738ed8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:05.015281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1304/2617] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:05.019939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720376b1-974d-4888-ac74-6ec7aee90d86 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.029355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720376b1-974d-4888-ac74-6ec7aee90d86 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:05.034581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bac17-30ce-4dda-b32e-b1e60dfb4680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:05.035308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bac17-30ce-4dda-b32e-b1e60dfb4680 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.035594 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bac17-30ce-4dda-b32e-b1e60dfb4680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.036026 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bac17-30ce-4dda-b32e-b1e60dfb4680 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.036717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bac17-30ce-4dda-b32e-b1e60dfb4680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:05.037022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bac17-30ce-4dda-b32e-b1e60dfb4680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:05.038597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bac17-30ce-4dda-b32e-b1e60dfb4680 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:05.050295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1314/2618] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:05.054493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de2e40-523f-4527-819d-660a1738ed8a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.063689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de2e40-523f-4527-819d-660a1738ed8a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:05.068317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e6c6d-97ab-4ad3-bc8d-7bbe007e0d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:05.068859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e6c6d-97ab-4ad3-bc8d-7bbe007e0d87 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.069140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e6c6d-97ab-4ad3-bc8d-7bbe007e0d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.069415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e6c6d-97ab-4ad3-bc8d-7bbe007e0d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.070073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e6c6d-97ab-4ad3-bc8d-7bbe007e0d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:05.072035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e6c6d-97ab-4ad3-bc8d-7bbe007e0d87 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:05.099548 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1305/2619] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:45:05 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:05.103655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bac17-30ce-4dda-b32e-b1e60dfb4680 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.109826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:05.115008 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bac17-30ce-4dda-b32e-b1e60dfb4680 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:05.171372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4ff8c-53be-482c-b39b-cd4b08f4446d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:05.171980 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4ff8c-53be-482c-b39b-cd4b08f4446d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.172262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4ff8c-53be-482c-b39b-cd4b08f4446d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.172637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4ff8c-53be-482c-b39b-cd4b08f4446d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.173952 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4ff8c-53be-482c-b39b-cd4b08f4446d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:05.175879 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4ff8c-53be-482c-b39b-cd4b08f4446d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:05.197006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1315/2620] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:45:05 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:45:05.201282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e6c6d-97ab-4ad3-bc8d-7bbe007e0d87 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.211845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e6c6d-97ab-4ad3-bc8d-7bbe007e0d87 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:05.216541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee9768-4ee3-47a2-b6f8-a0e060bd4f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:05.217145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee9768-4ee3-47a2-b6f8-a0e060bd4f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.217429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee9768-4ee3-47a2-b6f8-a0e060bd4f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.217705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee9768-4ee3-47a2-b6f8-a0e060bd4f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.218389 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee9768-4ee3-47a2-b6f8-a0e060bd4f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:05.220019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee9768-4ee3-47a2-b6f8-a0e060bd4f57 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:05.230686 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1306/2621] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:05.234653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4ff8c-53be-482c-b39b-cd4b08f4446d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.245363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4ff8c-53be-482c-b39b-cd4b08f4446d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:05.250351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7184b7a-c227-4f82-8b8c-5d5da687bde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:05.250943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7184b7a-c227-4f82-8b8c-5d5da687bde5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.251255 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7184b7a-c227-4f82-8b8c-5d5da687bde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.251559 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7184b7a-c227-4f82-8b8c-5d5da687bde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.252265 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7184b7a-c227-4f82-8b8c-5d5da687bde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:05.253822 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7184b7a-c227-4f82-8b8c-5d5da687bde5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:05.264097 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1316/2622] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:05.268541 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee9768-4ee3-47a2-b6f8-a0e060bd4f57 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.278468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee9768-4ee3-47a2-b6f8-a0e060bd4f57 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:05.283921 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26ab4c-c29c-406e-b865-b24e99d3aa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:05.284577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ab4c-c29c-406e-b865-b24e99d3aa66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.284862 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ab4c-c29c-406e-b865-b24e99d3aa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.285145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ab4c-c29c-406e-b865-b24e99d3aa66 admin admin] PATH_INFO: `/v3/projects/1d90df41ae714211ae5e28475b6bcaa6/users/f711b986230a4c48a891cba87b1ff518/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.300184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26ab4c-c29c-406e-b865-b24e99d3aa66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d90df41ae714211ae5e28475b6bcaa6, user_id=f711b986230a4c48a891cba87b1ff518, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:05.302835 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26ab4c-c29c-406e-b865-b24e99d3aa66 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:05.316104 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1307/2623] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:05 2022] PUT /identity/v3/projects/1d90df41ae714211ae5e28475b6bcaa6/users/f711b986230a4c48a891cba87b1ff518/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:05.327491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c95444-b3ac-40f6-9d94-cbaa59a09ba3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.327726 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c95444-b3ac-40f6-9d94-cbaa59a09ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.328004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c95444-b3ac-40f6-9d94-cbaa59a09ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.355781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-45c95444-b3ac-40f6-9d94-cbaa59a09ba3 None None] MFA Rules not processed for user `f711b986230a4c48a891cba87b1ff518`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:05.356998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-45c95444-b3ac-40f6-9d94-cbaa59a09ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:05.420093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1317/2624] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:05.424866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26ab4c-c29c-406e-b865-b24e99d3aa66 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.486281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26ab4c-c29c-406e-b865-b24e99d3aa66 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:05.490919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2697bed-73e4-420f-b17f-9b5f16c747dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:05.491568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2697bed-73e4-420f-b17f-9b5f16c747dd admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.491857 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2697bed-73e4-420f-b17f-9b5f16c747dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.492140 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2697bed-73e4-420f-b17f-9b5f16c747dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.492798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2697bed-73e4-420f-b17f-9b5f16c747dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:05.494740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2697bed-73e4-420f-b17f-9b5f16c747dd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:05.499191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:05.524766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1308/2625] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:45:05 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:05.528876 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c95444-b3ac-40f6-9d94-cbaa59a09ba3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.540175 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c95444-b3ac-40f6-9d94-cbaa59a09ba3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:05.595936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48021d-a19b-4e86-9cfe-029c5d1e2cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:05.596470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48021d-a19b-4e86-9cfe-029c5d1e2cff admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.596752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48021d-a19b-4e86-9cfe-029c5d1e2cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.597028 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48021d-a19b-4e86-9cfe-029c5d1e2cff admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.597693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48021d-a19b-4e86-9cfe-029c5d1e2cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:05.599616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48021d-a19b-4e86-9cfe-029c5d1e2cff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:05.619963 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1318/2626] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:45:05 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:05.623960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2697bed-73e4-420f-b17f-9b5f16c747dd admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.634902 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2697bed-73e4-420f-b17f-9b5f16c747dd admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:05.639456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99842e56-bc18-4aa5-91fe-e0a15900274d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:05.640057 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99842e56-bc18-4aa5-91fe-e0a15900274d admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.640334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99842e56-bc18-4aa5-91fe-e0a15900274d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.640608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99842e56-bc18-4aa5-91fe-e0a15900274d admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.641457 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99842e56-bc18-4aa5-91fe-e0a15900274d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:05.643775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99842e56-bc18-4aa5-91fe-e0a15900274d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:05.653782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1309/2627] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:05.658556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48021d-a19b-4e86-9cfe-029c5d1e2cff admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.663733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:05.669938 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48021d-a19b-4e86-9cfe-029c5d1e2cff admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:05.674788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc6865-5641-4367-8371-c3057e65a3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:05.675471 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc6865-5641-4367-8371-c3057e65a3f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.675754 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc6865-5641-4367-8371-c3057e65a3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.676030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc6865-5641-4367-8371-c3057e65a3f6 admin admin] PATH_INFO: `/v3/projects/4e1be0695cee43c5a03877c50b95c9b5/users/0165250d7b6e481d8572dce2998bac41/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.690363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc6865-5641-4367-8371-c3057e65a3f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1be0695cee43c5a03877c50b95c9b5, user_id=0165250d7b6e481d8572dce2998bac41, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:05.693017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc6865-5641-4367-8371-c3057e65a3f6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:05.706000 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1319/2628] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:05 2022] PUT /identity/v3/projects/4e1be0695cee43c5a03877c50b95c9b5/users/0165250d7b6e481d8572dce2998bac41/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:05.709903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99842e56-bc18-4aa5-91fe-e0a15900274d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.719931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99842e56-bc18-4aa5-91fe-e0a15900274d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:05.778073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb087d3-e4d3-47c0-a4de-f57a8e88a741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:05.778725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb087d3-e4d3-47c0-a4de-f57a8e88a741 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.778959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb087d3-e4d3-47c0-a4de-f57a8e88a741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.779270 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb087d3-e4d3-47c0-a4de-f57a8e88a741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.779955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb087d3-e4d3-47c0-a4de-f57a8e88a741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:05.781542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb087d3-e4d3-47c0-a4de-f57a8e88a741 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:05.791368 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1310/2629] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:05.795498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc6865-5641-4367-8371-c3057e65a3f6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.805766 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc6865-5641-4367-8371-c3057e65a3f6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:05.810280 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfe0b8-d51c-4ca9-ad6a-2a9b00825eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:05.811789 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfe0b8-d51c-4ca9-ad6a-2a9b00825eba admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.812078 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfe0b8-d51c-4ca9-ad6a-2a9b00825eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.812354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfe0b8-d51c-4ca9-ad6a-2a9b00825eba admin admin] PATH_INFO: `/v3/projects/4e1be0695cee43c5a03877c50b95c9b5/users/0165250d7b6e481d8572dce2998bac41/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.815014 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfe0b8-d51c-4ca9-ad6a-2a9b00825eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1be0695cee43c5a03877c50b95c9b5, user_id=0165250d7b6e481d8572dce2998bac41, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:05.817887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfe0b8-d51c-4ca9-ad6a-2a9b00825eba admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:05.827966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1320/2630] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:05 2022] PUT /identity/v3/projects/4e1be0695cee43c5a03877c50b95c9b5/users/0165250d7b6e481d8572dce2998bac41/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:05.838664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dac739-b5bd-48bf-917d-04273c3c6dad None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:05.838954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dac739-b5bd-48bf-917d-04273c3c6dad None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:05.839263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dac739-b5bd-48bf-917d-04273c3c6dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:05.866106 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-10dac739-b5bd-48bf-917d-04273c3c6dad None None] MFA Rules not processed for user `0165250d7b6e481d8572dce2998bac41`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:05.867269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-10dac739-b5bd-48bf-917d-04273c3c6dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:05.934210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1311/2631] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:05.945946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfe0b8-d51c-4ca9-ad6a-2a9b00825eba admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:05.947828 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c9dfe0b8-d51c-4ca9-ad6a-2a9b00825eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:06.006258 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfe0b8-d51c-4ca9-ad6a-2a9b00825eba admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:06.009900 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd5d10-f2c5-4ed3-977e-b7ccf5015a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:06.010501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd5d10-f2c5-4ed3-977e-b7ccf5015a00 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:06.010785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd5d10-f2c5-4ed3-977e-b7ccf5015a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:06.011149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd5d10-f2c5-4ed3-977e-b7ccf5015a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:06.019938 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd5d10-f2c5-4ed3-977e-b7ccf5015a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:06.021373 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd5d10-f2c5-4ed3-977e-b7ccf5015a00 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:06.030573 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1321/2632] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:06.113751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dac739-b5bd-48bf-917d-04273c3c6dad None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:06.123579 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dac739-b5bd-48bf-917d-04273c3c6dad None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:06.128114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092ae20-b2a1-43bd-9077-e1996b1afc81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:06.129516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092ae20-b2a1-43bd-9077-e1996b1afc81 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:06.129516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092ae20-b2a1-43bd-9077-e1996b1afc81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:06.129516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092ae20-b2a1-43bd-9077-e1996b1afc81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:06.183950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:06.188505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092ae20-b2a1-43bd-9077-e1996b1afc81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:06.190692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092ae20-b2a1-43bd-9077-e1996b1afc81 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:06.200164 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1312/2633] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:10.901853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd5d10-f2c5-4ed3-977e-b7ccf5015a00 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:11.004467 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd5d10-f2c5-4ed3-977e-b7ccf5015a00 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:11.008959 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:11.009598 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:11.009882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:11.010156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] PATH_INFO: `/v3/users/f711b986230a4c48a891cba87b1ff518` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:11.011375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=f711b986230a4c48a891cba87b1ff518)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:11.013313 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:11.041473 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:11.060068 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f711b986230a4c48a891cba87b1ff518'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.067617 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f711b986230a4c48a891cba87b1ff518'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.079084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f711b986230a4c48a891cba87b1ff518'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.080245 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f711b986230a4c48a891cba87b1ff518'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.080859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f711b986230a4c48a891cba87b1ff518'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.092579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1322/2634] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:10 2022] DELETE /identity/v3/users/f711b986230a4c48a891cba87b1ff518 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:11.097229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a832782-40fd-4b04-b959-99be7a6a91a4 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:11.097695 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a832782-40fd-4b04-b959-99be7a6a91a4 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:11.097819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a832782-40fd-4b04-b959-99be7a6a91a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:11.146663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3a832782-40fd-4b04-b959-99be7a6a91a4 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:11.147886 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3a832782-40fd-4b04-b959-99be7a6a91a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:11.205870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1313/2635] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:45:11.215564 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:11.217007 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:11.288646 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb218867-11ea-49e2-b16e-31fa2852f8ee admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:11.292184 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11317a2-fafb-4055-ad98-ff672195429f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:11.292791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11317a2-fafb-4055-ad98-ff672195429f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:11.293072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11317a2-fafb-4055-ad98-ff672195429f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:11.293347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11317a2-fafb-4055-ad98-ff672195429f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:11.302156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11317a2-fafb-4055-ad98-ff672195429f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:11.303621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11317a2-fafb-4055-ad98-ff672195429f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:11.312753 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1323/2636] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:11.584388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a832782-40fd-4b04-b959-99be7a6a91a4 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:11.585823 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3a832782-40fd-4b04-b959-99be7a6a91a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:11.599053 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a832782-40fd-4b04-b959-99be7a6a91a4 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:11.604055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:11.604749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:11.605523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:11.605815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] PATH_INFO: `/v3/projects/1d90df41ae714211ae5e28475b6bcaa6` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:11.607231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d90df41ae714211ae5e28475b6bcaa6)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:11.609115 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:11.656862 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d90df41ae714211ae5e28475b6bcaa6', 'request_id': 'req-84a2b424-1f0a-4139-8055-b6dbbdc065a7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.663311 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d90df41ae714211ae5e28475b6bcaa6', 'request_id': 'req-84a2b424-1f0a-4139-8055-b6dbbdc065a7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.669436 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d90df41ae714211ae5e28475b6bcaa6', 'request_id': 'req-84a2b424-1f0a-4139-8055-b6dbbdc065a7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.683633 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] The token cache is being invalidate because project 1d90df41ae714211ae5e28475b6bcaa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:11.683927 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.684562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.686070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1314/2637] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:11 2022] DELETE /identity/v3/projects/1d90df41ae714211ae5e28475b6bcaa6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:11.690037 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11317a2-fafb-4055-ad98-ff672195429f service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:11.699555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11317a2-fafb-4055-ad98-ff672195429f service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:11.756833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5e514-571e-4740-bc0d-4902492602cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:11.757480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5e514-571e-4740-bc0d-4902492602cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:11.758251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5e514-571e-4740-bc0d-4902492602cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:11.758540 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5e514-571e-4740-bc0d-4902492602cb admin admin] PATH_INFO: `/v3/users/0165250d7b6e481d8572dce2998bac41` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:11.759835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5e514-571e-4740-bc0d-4902492602cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0165250d7b6e481d8572dce2998bac41)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:11.761845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5e514-571e-4740-bc0d-4902492602cb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:11.799669 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:11.810086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e5b5e514-571e-4740-bc0d-4902492602cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0165250d7b6e481d8572dce2998bac41'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.817487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e5b5e514-571e-4740-bc0d-4902492602cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0165250d7b6e481d8572dce2998bac41'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.830084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e5b5e514-571e-4740-bc0d-4902492602cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0165250d7b6e481d8572dce2998bac41'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.830675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e5b5e514-571e-4740-bc0d-4902492602cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0165250d7b6e481d8572dce2998bac41'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.831320 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e5b5e514-571e-4740-bc0d-4902492602cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0165250d7b6e481d8572dce2998bac41'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:11.843350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1324/2638] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:11 2022] DELETE /identity/v3/users/0165250d7b6e481d8572dce2998bac41 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:45:12.118754 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.120234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:12.133869 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2b424-1f0a-4139-8055-b6dbbdc065a7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.180811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:12.192191 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a907bdd-5734-4159-a3e7-f9917fae48fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.192861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a907bdd-5734-4159-a3e7-f9917fae48fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.193149 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a907bdd-5734-4159-a3e7-f9917fae48fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.193430 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a907bdd-5734-4159-a3e7-f9917fae48fe admin admin] PATH_INFO: `/v3/projects/4e1be0695cee43c5a03877c50b95c9b5` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.194587 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a907bdd-5734-4159-a3e7-f9917fae48fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e1be0695cee43c5a03877c50b95c9b5)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.196540 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a907bdd-5734-4159-a3e7-f9917fae48fe admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.242665 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5a907bdd-5734-4159-a3e7-f9917fae48fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e1be0695cee43c5a03877c50b95c9b5', 'request_id': 'req-5a907bdd-5734-4159-a3e7-f9917fae48fe'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:12.248019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5a907bdd-5734-4159-a3e7-f9917fae48fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e1be0695cee43c5a03877c50b95c9b5', 'request_id': 'req-5a907bdd-5734-4159-a3e7-f9917fae48fe'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:12.254371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5a907bdd-5734-4159-a3e7-f9917fae48fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e1be0695cee43c5a03877c50b95c9b5', 'request_id': 'req-5a907bdd-5734-4159-a3e7-f9917fae48fe'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:12.266328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5a907bdd-5734-4159-a3e7-f9917fae48fe admin admin] The token cache is being invalidate because project 4e1be0695cee43c5a03877c50b95c9b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:12.267357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5a907bdd-5734-4159-a3e7-f9917fae48fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:12.268604 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5a907bdd-5734-4159-a3e7-f9917fae48fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:12.270234 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1315/2639] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:12 2022] DELETE /identity/v3/projects/4e1be0695cee43c5a03877c50b95c9b5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:12.275287 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164d549-9c7e-4947-b416-6dbbc34154d1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.275751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164d549-9c7e-4947-b416-6dbbc34154d1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.275849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164d549-9c7e-4947-b416-6dbbc34154d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.322629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4164d549-9c7e-4947-b416-6dbbc34154d1 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:12.323857 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4164d549-9c7e-4947-b416-6dbbc34154d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:12.381411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1325/2640] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:12.388772 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d005858-498b-4b4c-98f0-fb6702a605e6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.389054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d005858-498b-4b4c-98f0-fb6702a605e6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.389358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d005858-498b-4b4c-98f0-fb6702a605e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.409919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8d005858-498b-4b4c-98f0-fb6702a605e6 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:12.411153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8d005858-498b-4b4c-98f0-fb6702a605e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:12.418822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1316/2641] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:12.423060 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164d549-9c7e-4947-b416-6dbbc34154d1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.432740 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164d549-9c7e-4947-b416-6dbbc34154d1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.437304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900f331-e64b-4f28-87d4-366bf256e13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.438003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f331-e64b-4f28-87d4-366bf256e13f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.438291 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f331-e64b-4f28-87d4-366bf256e13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.438784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f331-e64b-4f28-87d4-366bf256e13f admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.439506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900f331-e64b-4f28-87d4-366bf256e13f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:12.439831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900f331-e64b-4f28-87d4-366bf256e13f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.441418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900f331-e64b-4f28-87d4-366bf256e13f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.452849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1326/2642] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:12.458825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d005858-498b-4b4c-98f0-fb6702a605e6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.468275 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d005858-498b-4b4c-98f0-fb6702a605e6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.474118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239462b-d547-4a8b-962d-9a639191db73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.474721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239462b-d547-4a8b-962d-9a639191db73 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.475004 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239462b-d547-4a8b-962d-9a639191db73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.475326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239462b-d547-4a8b-962d-9a639191db73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.475998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239462b-d547-4a8b-962d-9a639191db73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.477888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239462b-d547-4a8b-962d-9a639191db73 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.512922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1317/2643] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:45:12 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:12.517718 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900f331-e64b-4f28-87d4-366bf256e13f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.528181 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900f331-e64b-4f28-87d4-366bf256e13f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.585096 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66920ee0-907c-4958-80de-fac4cd584aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.585709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66920ee0-907c-4958-80de-fac4cd584aef admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.585991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66920ee0-907c-4958-80de-fac4cd584aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.586268 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66920ee0-907c-4958-80de-fac4cd584aef admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.586919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66920ee0-907c-4958-80de-fac4cd584aef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.588851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66920ee0-907c-4958-80de-fac4cd584aef admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.609572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1327/2644] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:45:12 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:12.613890 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239462b-d547-4a8b-962d-9a639191db73 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.629825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239462b-d547-4a8b-962d-9a639191db73 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.635420 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6469a-fc2c-4bf3-8593-9abc8884943f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.635973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6469a-fc2c-4bf3-8593-9abc8884943f admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.636254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6469a-fc2c-4bf3-8593-9abc8884943f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.636528 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6469a-fc2c-4bf3-8593-9abc8884943f admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.637215 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6469a-fc2c-4bf3-8593-9abc8884943f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.638811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6469a-fc2c-4bf3-8593-9abc8884943f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.649623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1318/2645] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:12.653613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66920ee0-907c-4958-80de-fac4cd584aef admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.660602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:12.664835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66920ee0-907c-4958-80de-fac4cd584aef admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.670890 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1387e9-31c2-4ea7-a7c7-697f88da7159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.671534 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1387e9-31c2-4ea7-a7c7-697f88da7159 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.671867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1387e9-31c2-4ea7-a7c7-697f88da7159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.672298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1387e9-31c2-4ea7-a7c7-697f88da7159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.673093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1387e9-31c2-4ea7-a7c7-697f88da7159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.675017 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1387e9-31c2-4ea7-a7c7-697f88da7159 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.685472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1328/2646] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:12.690054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6469a-fc2c-4bf3-8593-9abc8884943f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.699950 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6469a-fc2c-4bf3-8593-9abc8884943f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.705030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de3de0-090d-4dfb-bbbf-d8a107689465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.705679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de3de0-090d-4dfb-bbbf-d8a107689465 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.705966 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de3de0-090d-4dfb-bbbf-d8a107689465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.706239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de3de0-090d-4dfb-bbbf-d8a107689465 admin admin] PATH_INFO: `/v3/projects/3b0347c9c5004f189654a51330ea809e/users/100d772b58d344369a3800789b3a1bc8/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.720644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de3de0-090d-4dfb-bbbf-d8a107689465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0347c9c5004f189654a51330ea809e, user_id=100d772b58d344369a3800789b3a1bc8, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.723341 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de3de0-090d-4dfb-bbbf-d8a107689465 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.736426 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1319/2647] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:12 2022] PUT /identity/v3/projects/3b0347c9c5004f189654a51330ea809e/users/100d772b58d344369a3800789b3a1bc8/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:12.747623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569c9b9-eabf-43eb-8944-3fda2a47143a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.747915 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569c9b9-eabf-43eb-8944-3fda2a47143a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.748261 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569c9b9-eabf-43eb-8944-3fda2a47143a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.778820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3569c9b9-eabf-43eb-8944-3fda2a47143a None None] MFA Rules not processed for user `100d772b58d344369a3800789b3a1bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:12.780083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3569c9b9-eabf-43eb-8944-3fda2a47143a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:12.842648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1329/2648] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:12.847293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de3de0-090d-4dfb-bbbf-d8a107689465 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.907828 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de3de0-090d-4dfb-bbbf-d8a107689465 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:12.912371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15f693-5b4a-432e-889d-9439f5b785b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:12.912980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15f693-5b4a-432e-889d-9439f5b785b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:12.913262 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15f693-5b4a-432e-889d-9439f5b785b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:12.913554 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15f693-5b4a-432e-889d-9439f5b785b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:12.914221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15f693-5b4a-432e-889d-9439f5b785b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:12.916157 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15f693-5b4a-432e-889d-9439f5b785b6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:12.943680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1320/2649] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:45:12 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:12.948363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569c9b9-eabf-43eb-8944-3fda2a47143a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:12.959529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569c9b9-eabf-43eb-8944-3fda2a47143a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:13.014995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f021d-7295-48d0-a0d0-cba12da625e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:13.015636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f021d-7295-48d0-a0d0-cba12da625e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:13.015918 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f021d-7295-48d0-a0d0-cba12da625e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:13.016190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f021d-7295-48d0-a0d0-cba12da625e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:13.016994 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f021d-7295-48d0-a0d0-cba12da625e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:13.018867 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f021d-7295-48d0-a0d0-cba12da625e0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:13.039022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1330/2650] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:45:12 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:13.043099 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15f693-5b4a-432e-889d-9439f5b785b6 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:13.049034 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:13.055391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15f693-5b4a-432e-889d-9439f5b785b6 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:13.061042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ac092-df27-4a0d-9355-e53420813633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:13.061649 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ac092-df27-4a0d-9355-e53420813633 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:13.061928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ac092-df27-4a0d-9355-e53420813633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:13.062204 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ac092-df27-4a0d-9355-e53420813633 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:13.062892 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ac092-df27-4a0d-9355-e53420813633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:13.065394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ac092-df27-4a0d-9355-e53420813633 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:13.075848 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1321/2651] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:13.080108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f021d-7295-48d0-a0d0-cba12da625e0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:13.089808 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f021d-7295-48d0-a0d0-cba12da625e0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:13.094272 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f36508-e9ab-44da-99ad-5492056e8898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:13.095031 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f36508-e9ab-44da-99ad-5492056e8898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:13.095342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f36508-e9ab-44da-99ad-5492056e8898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:13.095619 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f36508-e9ab-44da-99ad-5492056e8898 admin admin] PATH_INFO: `/v3/projects/5630a37a24de4e93b2c93a916de54eef/users/d28b1d279bd94e4ab965e16174a376ec/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:13.109626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f36508-e9ab-44da-99ad-5492056e8898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5630a37a24de4e93b2c93a916de54eef, user_id=d28b1d279bd94e4ab965e16174a376ec, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:13.112812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f36508-e9ab-44da-99ad-5492056e8898 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:13.126114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1331/2652] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:13 2022] PUT /identity/v3/projects/5630a37a24de4e93b2c93a916de54eef/users/d28b1d279bd94e4ab965e16174a376ec/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:45:13.130799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ac092-df27-4a0d-9355-e53420813633 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:13.140778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ac092-df27-4a0d-9355-e53420813633 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:13.196866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6428094-6c51-4065-91eb-75f1f42588e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:13.197479 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6428094-6c51-4065-91eb-75f1f42588e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:13.197758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6428094-6c51-4065-91eb-75f1f42588e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:13.198031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6428094-6c51-4065-91eb-75f1f42588e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:13.198706 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6428094-6c51-4065-91eb-75f1f42588e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:13.200327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6428094-6c51-4065-91eb-75f1f42588e4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:13.209907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1322/2653] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:13.214053 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f36508-e9ab-44da-99ad-5492056e8898 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:13.224525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f36508-e9ab-44da-99ad-5492056e8898 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:13.229243 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b247199-8004-47e8-80e3-032ba1c74e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:13.229895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b247199-8004-47e8-80e3-032ba1c74e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:13.230183 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b247199-8004-47e8-80e3-032ba1c74e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:13.230472 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b247199-8004-47e8-80e3-032ba1c74e17 admin admin] PATH_INFO: `/v3/projects/5630a37a24de4e93b2c93a916de54eef/users/d28b1d279bd94e4ab965e16174a376ec/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:13.232616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b247199-8004-47e8-80e3-032ba1c74e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5630a37a24de4e93b2c93a916de54eef, user_id=d28b1d279bd94e4ab965e16174a376ec, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:13.235280 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b247199-8004-47e8-80e3-032ba1c74e17 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:13.246159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1332/2654] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:13 2022] PUT /identity/v3/projects/5630a37a24de4e93b2c93a916de54eef/users/d28b1d279bd94e4ab965e16174a376ec/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:13.257210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e1f4a-99f5-489c-a60a-f843dd2a7311 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:13.257499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e1f4a-99f5-489c-a60a-f843dd2a7311 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:13.257774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e1f4a-99f5-489c-a60a-f843dd2a7311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:13.285378 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5b2e1f4a-99f5-489c-a60a-f843dd2a7311 None None] MFA Rules not processed for user `d28b1d279bd94e4ab965e16174a376ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:13.286610 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5b2e1f4a-99f5-489c-a60a-f843dd2a7311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:13.352664 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1323/2655] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:13.363917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b247199-8004-47e8-80e3-032ba1c74e17 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:13.365678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b247199-8004-47e8-80e3-032ba1c74e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:13.406335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:13.426999 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b247199-8004-47e8-80e3-032ba1c74e17 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:13.430613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ee1a0-6fd1-4a8b-83a9-2e75940f04f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:13.431256 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ee1a0-6fd1-4a8b-83a9-2e75940f04f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:13.431538 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ee1a0-6fd1-4a8b-83a9-2e75940f04f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:13.431810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ee1a0-6fd1-4a8b-83a9-2e75940f04f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:13.440784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ee1a0-6fd1-4a8b-83a9-2e75940f04f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:13.442203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ee1a0-6fd1-4a8b-83a9-2e75940f04f6 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:13.452254 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1333/2656] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:13.847880 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e1f4a-99f5-489c-a60a-f843dd2a7311 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:13.911192 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e1f4a-99f5-489c-a60a-f843dd2a7311 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:13.915646 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e25438-43c6-47d9-a8bf-39dc4cecef41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:13.916282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e25438-43c6-47d9-a8bf-39dc4cecef41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:13.916566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e25438-43c6-47d9-a8bf-39dc4cecef41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:13.916846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e25438-43c6-47d9-a8bf-39dc4cecef41 admin admin] PATH_INFO: `/v3/users/100d772b58d344369a3800789b3a1bc8` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:13.917997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e25438-43c6-47d9-a8bf-39dc4cecef41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=100d772b58d344369a3800789b3a1bc8)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:13.919955 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e25438-43c6-47d9-a8bf-39dc4cecef41 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:13.934779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:13.974697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b3e25438-43c6-47d9-a8bf-39dc4cecef41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100d772b58d344369a3800789b3a1bc8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:13.987816 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b3e25438-43c6-47d9-a8bf-39dc4cecef41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100d772b58d344369a3800789b3a1bc8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:13.988495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b3e25438-43c6-47d9-a8bf-39dc4cecef41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100d772b58d344369a3800789b3a1bc8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:13.995319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b3e25438-43c6-47d9-a8bf-39dc4cecef41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100d772b58d344369a3800789b3a1bc8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:14.004373 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b3e25438-43c6-47d9-a8bf-39dc4cecef41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100d772b58d344369a3800789b3a1bc8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:14.004772 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ee1a0-6fd1-4a8b-83a9-2e75940f04f6 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:14.006403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fa2ee1a0-6fd1-4a8b-83a9-2e75940f04f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:14.011716 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1324/2657] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:13 2022] DELETE /identity/v3/users/100d772b58d344369a3800789b3a1bc8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:14.015651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6e6dc-375a-4c93-9f6b-33c83984f76d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:14.015943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6e6dc-375a-4c93-9f6b-33c83984f76d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:14.016231 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6e6dc-375a-4c93-9f6b-33c83984f76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:14.035900 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a0b6e6dc-375a-4c93-9f6b-33c83984f76d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:14.037150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a0b6e6dc-375a-4c93-9f6b-33c83984f76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:14.101080 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ee1a0-6fd1-4a8b-83a9-2e75940f04f6 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:14.101785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1325/2658] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:45:14.112550 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6e6dc-375a-4c93-9f6b-33c83984f76d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:14.113922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a0b6e6dc-375a-4c93-9f6b-33c83984f76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:14.165330 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b704d7-d2dd-4db0-8957-9fa9e48b2e3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928c5d1284b74f1291c031323bb03a20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:14.166189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b704d7-d2dd-4db0-8957-9fa9e48b2e3b service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:14.166652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b704d7-d2dd-4db0-8957-9fa9e48b2e3b service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:14.166938 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b704d7-d2dd-4db0-8957-9fa9e48b2e3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:14.168756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-22b704d7-d2dd-4db0-8957-9fa9e48b2e3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:14.172120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6e6dc-375a-4c93-9f6b-33c83984f76d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:14.176477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc849b-4a58-4849-9708-6f275af50fab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:14.177093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc849b-4a58-4849-9708-6f275af50fab service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:14.177376 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc849b-4a58-4849-9708-6f275af50fab service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:14.177650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc849b-4a58-4849-9708-6f275af50fab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:14.181630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b704d7-d2dd-4db0-8957-9fa9e48b2e3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:14.183243 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b704d7-d2dd-4db0-8957-9fa9e48b2e3b service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:14.186659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc849b-4a58-4849-9708-6f275af50fab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:14.188118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc849b-4a58-4849-9708-6f275af50fab service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:14.193762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1334/2659] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:45:14.198176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1326/2660] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:45:14.468630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b704d7-d2dd-4db0-8957-9fa9e48b2e3b service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:14.471209 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-22b704d7-d2dd-4db0-8957-9fa9e48b2e3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:14.486513 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b704d7-d2dd-4db0-8957-9fa9e48b2e3b service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:14.491392 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:14.492060 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:14.492346 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:14.492626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] PATH_INFO: `/v3/projects/3b0347c9c5004f189654a51330ea809e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:14.493792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0347c9c5004f189654a51330ea809e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:14.495789 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:14.541043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:14.544359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0347c9c5004f189654a51330ea809e', 'request_id': 'req-d286b5a5-7904-495b-a3fd-db42f4fe8932'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:14.552673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0347c9c5004f189654a51330ea809e', 'request_id': 'req-d286b5a5-7904-495b-a3fd-db42f4fe8932'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:14.558322 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0347c9c5004f189654a51330ea809e', 'request_id': 'req-d286b5a5-7904-495b-a3fd-db42f4fe8932'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:14.569717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] The token cache is being invalidate because project 3b0347c9c5004f189654a51330ea809e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:14.570002 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:14.570636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:14.572160 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1335/2661] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:14 2022] DELETE /identity/v3/projects/3b0347c9c5004f189654a51330ea809e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:14.576246 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc849b-4a58-4849-9708-6f275af50fab service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:14.585904 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc849b-4a58-4849-9708-6f275af50fab service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:14.619322 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:14.645919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab06d43-1f5d-4a81-a865-6ed1422f64cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:14.646622 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab06d43-1f5d-4a81-a865-6ed1422f64cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:14.646903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab06d43-1f5d-4a81-a865-6ed1422f64cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:14.647342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab06d43-1f5d-4a81-a865-6ed1422f64cc admin admin] PATH_INFO: `/v3/users/d28b1d279bd94e4ab965e16174a376ec` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:14.648515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab06d43-1f5d-4a81-a865-6ed1422f64cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28b1d279bd94e4ab965e16174a376ec)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:14.651059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab06d43-1f5d-4a81-a865-6ed1422f64cc admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:14.698778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fab06d43-1f5d-4a81-a865-6ed1422f64cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28b1d279bd94e4ab965e16174a376ec'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:14.711118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fab06d43-1f5d-4a81-a865-6ed1422f64cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28b1d279bd94e4ab965e16174a376ec'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:14.711782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fab06d43-1f5d-4a81-a865-6ed1422f64cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28b1d279bd94e4ab965e16174a376ec'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:14.718551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fab06d43-1f5d-4a81-a865-6ed1422f64cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28b1d279bd94e4ab965e16174a376ec'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:14.726369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-fab06d43-1f5d-4a81-a865-6ed1422f64cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28b1d279bd94e4ab965e16174a376ec'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:14.732381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1327/2662] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:14 2022] DELETE /identity/v3/users/d28b1d279bd94e4ab965e16174a376ec => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:15.023709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.025592 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:15.042329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286b5a5-7904-495b-a3fd-db42f4fe8932 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.105583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10026c-96e7-4df3-a067-f4f6e22941ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.106318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10026c-96e7-4df3-a067-f4f6e22941ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.106677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10026c-96e7-4df3-a067-f4f6e22941ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.106969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10026c-96e7-4df3-a067-f4f6e22941ca admin admin] PATH_INFO: `/v3/projects/5630a37a24de4e93b2c93a916de54eef` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.108276 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10026c-96e7-4df3-a067-f4f6e22941ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=5630a37a24de4e93b2c93a916de54eef)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.110381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10026c-96e7-4df3-a067-f4f6e22941ca admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.166973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-af10026c-96e7-4df3-a067-f4f6e22941ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5630a37a24de4e93b2c93a916de54eef', 'request_id': 'req-af10026c-96e7-4df3-a067-f4f6e22941ca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:15.177189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-af10026c-96e7-4df3-a067-f4f6e22941ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5630a37a24de4e93b2c93a916de54eef', 'request_id': 'req-af10026c-96e7-4df3-a067-f4f6e22941ca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:15.184584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-af10026c-96e7-4df3-a067-f4f6e22941ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5630a37a24de4e93b2c93a916de54eef', 'request_id': 'req-af10026c-96e7-4df3-a067-f4f6e22941ca'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:15.197648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-af10026c-96e7-4df3-a067-f4f6e22941ca admin admin] The token cache is being invalidate because project 5630a37a24de4e93b2c93a916de54eef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:15.198029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-af10026c-96e7-4df3-a067-f4f6e22941ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:15.198770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-af10026c-96e7-4df3-a067-f4f6e22941ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:15.200397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1336/2663] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:15 2022] DELETE /identity/v3/projects/5630a37a24de4e93b2c93a916de54eef => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:15.205536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0efbd-974e-4d6f-a597-f9d9448ed675 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.206006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0efbd-974e-4d6f-a597-f9d9448ed675 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.206126 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0efbd-974e-4d6f-a597-f9d9448ed675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.226662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-16b0efbd-974e-4d6f-a597-f9d9448ed675 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:15.227970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-16b0efbd-974e-4d6f-a597-f9d9448ed675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:15.290408 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1328/2664] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:15.300124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a08ac-7088-44ef-9eeb-6ccd12517bcc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.300413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a08ac-7088-44ef-9eeb-6ccd12517bcc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.300698 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a08ac-7088-44ef-9eeb-6ccd12517bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.320483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-063a08ac-7088-44ef-9eeb-6ccd12517bcc None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:15.321693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-063a08ac-7088-44ef-9eeb-6ccd12517bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:15.329593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1337/2665] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:15.333942 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0efbd-974e-4d6f-a597-f9d9448ed675 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.343605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0efbd-974e-4d6f-a597-f9d9448ed675 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.348271 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa90f8-58b4-4e7c-acdc-4dc1771c00dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.348867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa90f8-58b4-4e7c-acdc-4dc1771c00dc admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.349146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa90f8-58b4-4e7c-acdc-4dc1771c00dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.349419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa90f8-58b4-4e7c-acdc-4dc1771c00dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.350109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa90f8-58b4-4e7c-acdc-4dc1771c00dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:15.350439 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa90f8-58b4-4e7c-acdc-4dc1771c00dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.352078 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa90f8-58b4-4e7c-acdc-4dc1771c00dc admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.364186 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1329/2666] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:15.368659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a08ac-7088-44ef-9eeb-6ccd12517bcc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.378656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a08ac-7088-44ef-9eeb-6ccd12517bcc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.383313 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8aeb4d-2df5-4f08-b06d-314dc48bf108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.383916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8aeb4d-2df5-4f08-b06d-314dc48bf108 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.384201 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8aeb4d-2df5-4f08-b06d-314dc48bf108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.384479 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8aeb4d-2df5-4f08-b06d-314dc48bf108 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.385172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8aeb4d-2df5-4f08-b06d-314dc48bf108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.387086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8aeb4d-2df5-4f08-b06d-314dc48bf108 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.414875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1338/2667] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:45:15 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:45:15.419045 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa90f8-58b4-4e7c-acdc-4dc1771c00dc admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.429721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa90f8-58b4-4e7c-acdc-4dc1771c00dc admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.470549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:15.491606 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1304b-18fa-4c80-9b25-9a58b335a02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.492230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1304b-18fa-4c80-9b25-9a58b335a02d admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.492512 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1304b-18fa-4c80-9b25-9a58b335a02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.492786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1304b-18fa-4c80-9b25-9a58b335a02d admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.493626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1304b-18fa-4c80-9b25-9a58b335a02d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.495717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1304b-18fa-4c80-9b25-9a58b335a02d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.518870 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1330/2668] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:45:15 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:15.522988 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8aeb4d-2df5-4f08-b06d-314dc48bf108 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.534067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8aeb4d-2df5-4f08-b06d-314dc48bf108 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.538907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48c0fb-fb2f-4456-b4fc-5abe35053eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.539562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48c0fb-fb2f-4456-b4fc-5abe35053eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.539906 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48c0fb-fb2f-4456-b4fc-5abe35053eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.540186 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48c0fb-fb2f-4456-b4fc-5abe35053eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.540949 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48c0fb-fb2f-4456-b4fc-5abe35053eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.542618 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48c0fb-fb2f-4456-b4fc-5abe35053eb3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.553794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1339/2669] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:15.557801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1304b-18fa-4c80-9b25-9a58b335a02d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.569160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1304b-18fa-4c80-9b25-9a58b335a02d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.574064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4afed-53a2-4040-9a93-b4512912fb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.574810 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4afed-53a2-4040-9a93-b4512912fb4c admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.575135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4afed-53a2-4040-9a93-b4512912fb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.575422 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4afed-53a2-4040-9a93-b4512912fb4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.576114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4afed-53a2-4040-9a93-b4512912fb4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.577705 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4afed-53a2-4040-9a93-b4512912fb4c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.589058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1331/2670] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:15.593508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48c0fb-fb2f-4456-b4fc-5abe35053eb3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.604742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48c0fb-fb2f-4456-b4fc-5abe35053eb3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.609455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f5209-7cca-494f-8f9b-95f7b3330a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.610138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f5209-7cca-494f-8f9b-95f7b3330a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.611636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f5209-7cca-494f-8f9b-95f7b3330a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.611923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f5209-7cca-494f-8f9b-95f7b3330a3f admin admin] PATH_INFO: `/v3/projects/0138c80539a040e29683f3534e2e7139/users/89eccd7d264e40d8bb908c653c795b99/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.627485 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f5209-7cca-494f-8f9b-95f7b3330a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0138c80539a040e29683f3534e2e7139, user_id=89eccd7d264e40d8bb908c653c795b99, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.630154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f5209-7cca-494f-8f9b-95f7b3330a3f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.638610 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:15.644603 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1340/2671] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:15 2022] PUT /identity/v3/projects/0138c80539a040e29683f3534e2e7139/users/89eccd7d264e40d8bb908c653c795b99/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:15.711238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f2ae5-293f-4fc5-ab33-3ff5e8808b05 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.711523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f2ae5-293f-4fc5-ab33-3ff5e8808b05 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.711802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f2ae5-293f-4fc5-ab33-3ff5e8808b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.738574 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-824f2ae5-293f-4fc5-ab33-3ff5e8808b05 None None] MFA Rules not processed for user `89eccd7d264e40d8bb908c653c795b99`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:15.739803 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-824f2ae5-293f-4fc5-ab33-3ff5e8808b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:15.804650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1332/2672] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:15.815978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f5209-7cca-494f-8f9b-95f7b3330a3f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.817698 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-156f5209-7cca-494f-8f9b-95f7b3330a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:15.873529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f5209-7cca-494f-8f9b-95f7b3330a3f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.878159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38602970-67e7-4c4c-94e2-55e23384d9d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.878761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38602970-67e7-4c4c-94e2-55e23384d9d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.879041 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38602970-67e7-4c4c-94e2-55e23384d9d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.879349 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38602970-67e7-4c4c-94e2-55e23384d9d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.887629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38602970-67e7-4c4c-94e2-55e23384d9d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.889043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38602970-67e7-4c4c-94e2-55e23384d9d9 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:15.898304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1341/2673] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:15.919712 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f2ae5-293f-4fc5-ab33-3ff5e8808b05 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:15.984039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f2ae5-293f-4fc5-ab33-3ff5e8808b05 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:15.989603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:15.990241 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:15.990529 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:15.990808 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] PATH_INFO: `/v3/users/89eccd7d264e40d8bb908c653c795b99` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:15.992021 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89eccd7d264e40d8bb908c653c795b99)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:15.993946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:16.038535 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89eccd7d264e40d8bb908c653c795b99'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:16.042884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:16.052246 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89eccd7d264e40d8bb908c653c795b99'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:16.053084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89eccd7d264e40d8bb908c653c795b99'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:16.060443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89eccd7d264e40d8bb908c653c795b99'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:16.067923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89eccd7d264e40d8bb908c653c795b99'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:16.073956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1333/2674] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:15 2022] DELETE /identity/v3/users/89eccd7d264e40d8bb908c653c795b99 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:16.078468 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87e968-71d1-435d-804e-8b7b2b62eb76 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.078756 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87e968-71d1-435d-804e-8b7b2b62eb76 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.079295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87e968-71d1-435d-804e-8b7b2b62eb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.097446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8b87e968-71d1-435d-804e-8b7b2b62eb76 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:16.098636 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b87e968-71d1-435d-804e-8b7b2b62eb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:16.158095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1342/2675] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:16.167924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:16.169328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:16.227375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b889437-a2d1-4559-ae40-e28c081362e1 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:16.231343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47000130-69ce-43d2-86dd-8147e3968374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:16.231969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47000130-69ce-43d2-86dd-8147e3968374 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.232258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47000130-69ce-43d2-86dd-8147e3968374 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.232538 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47000130-69ce-43d2-86dd-8147e3968374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.242632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47000130-69ce-43d2-86dd-8147e3968374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:16.244114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47000130-69ce-43d2-86dd-8147e3968374 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:16.254719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1334/2676] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:16.525852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87e968-71d1-435d-804e-8b7b2b62eb76 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:16.527470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b87e968-71d1-435d-804e-8b7b2b62eb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:16.541070 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87e968-71d1-435d-804e-8b7b2b62eb76 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:16.545836 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503842c-2d63-4942-b690-babd4e1451e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:16.546480 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503842c-2d63-4942-b690-babd4e1451e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.546769 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503842c-2d63-4942-b690-babd4e1451e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.547130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503842c-2d63-4942-b690-babd4e1451e9 admin admin] PATH_INFO: `/v3/projects/0138c80539a040e29683f3534e2e7139` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.549099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503842c-2d63-4942-b690-babd4e1451e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0138c80539a040e29683f3534e2e7139)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:16.551205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503842c-2d63-4942-b690-babd4e1451e9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:16.597309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9503842c-2d63-4942-b690-babd4e1451e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0138c80539a040e29683f3534e2e7139', 'request_id': 'req-9503842c-2d63-4942-b690-babd4e1451e9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:16.604522 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9503842c-2d63-4942-b690-babd4e1451e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0138c80539a040e29683f3534e2e7139', 'request_id': 'req-9503842c-2d63-4942-b690-babd4e1451e9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:16.610904 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9503842c-2d63-4942-b690-babd4e1451e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0138c80539a040e29683f3534e2e7139', 'request_id': 'req-9503842c-2d63-4942-b690-babd4e1451e9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:16.621936 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9503842c-2d63-4942-b690-babd4e1451e9 admin admin] The token cache is being invalidate because project 0138c80539a040e29683f3534e2e7139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:16.622230 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9503842c-2d63-4942-b690-babd4e1451e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:16.622864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9503842c-2d63-4942-b690-babd4e1451e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:16.624664 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1343/2677] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:16 2022] DELETE /identity/v3/projects/0138c80539a040e29683f3534e2e7139 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:16.629602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f13685-70fc-477c-aa29-1f95eacd0084 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.629899 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f13685-70fc-477c-aa29-1f95eacd0084 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.630181 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f13685-70fc-477c-aa29-1f95eacd0084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.650131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-13f13685-70fc-477c-aa29-1f95eacd0084 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:16.651386 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-13f13685-70fc-477c-aa29-1f95eacd0084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:16.717544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1335/2678] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:16.725874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6a501-d183-4963-ac01-2fad33f4f6ea None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.726164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6a501-d183-4963-ac01-2fad33f4f6ea None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.726441 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6a501-d183-4963-ac01-2fad33f4f6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.745539 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-84e6a501-d183-4963-ac01-2fad33f4f6ea None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:16.746807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-84e6a501-d183-4963-ac01-2fad33f4f6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:16.755028 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1344/2679] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:16.759155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f13685-70fc-477c-aa29-1f95eacd0084 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:16.769675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f13685-70fc-477c-aa29-1f95eacd0084 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:16.774356 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393afa3f-3013-4aa8-bf8d-64189550b85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:16.774968 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393afa3f-3013-4aa8-bf8d-64189550b85d admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.775277 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393afa3f-3013-4aa8-bf8d-64189550b85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.775559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393afa3f-3013-4aa8-bf8d-64189550b85d admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.776251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393afa3f-3013-4aa8-bf8d-64189550b85d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:16.776561 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393afa3f-3013-4aa8-bf8d-64189550b85d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:16.778165 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393afa3f-3013-4aa8-bf8d-64189550b85d admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:16.790400 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1336/2680] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:16.794607 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6a501-d183-4963-ac01-2fad33f4f6ea None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:16.803630 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6a501-d183-4963-ac01-2fad33f4f6ea None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:16.808171 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d50e54-2e6c-492d-9de6-88bc0413a111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:16.808767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d50e54-2e6c-492d-9de6-88bc0413a111 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.809048 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d50e54-2e6c-492d-9de6-88bc0413a111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.809321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d50e54-2e6c-492d-9de6-88bc0413a111 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.809986 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d50e54-2e6c-492d-9de6-88bc0413a111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:16.811918 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d50e54-2e6c-492d-9de6-88bc0413a111 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:16.833030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:16.842157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1345/2681] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:45:16 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:16.847507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393afa3f-3013-4aa8-bf8d-64189550b85d admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:16.857396 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393afa3f-3013-4aa8-bf8d-64189550b85d admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:16.913543 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d70b0d-265d-4ee8-9f1b-7a85b87afefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:16.914154 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d70b0d-265d-4ee8-9f1b-7a85b87afefd admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.914469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d70b0d-265d-4ee8-9f1b-7a85b87afefd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.914750 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d70b0d-265d-4ee8-9f1b-7a85b87afefd admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.915438 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d70b0d-265d-4ee8-9f1b-7a85b87afefd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:16.917397 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d70b0d-265d-4ee8-9f1b-7a85b87afefd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:16.925333 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:16.940094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1337/2682] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:45:16 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:16.944275 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d50e54-2e6c-492d-9de6-88bc0413a111 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:16.955517 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d50e54-2e6c-492d-9de6-88bc0413a111 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:16.960759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8ccab-7467-4e31-9f93-b2cd8fbb77ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:16.961374 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8ccab-7467-4e31-9f93-b2cd8fbb77ca admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.961663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8ccab-7467-4e31-9f93-b2cd8fbb77ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.961951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8ccab-7467-4e31-9f93-b2cd8fbb77ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.962674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8ccab-7467-4e31-9f93-b2cd8fbb77ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:16.964291 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8ccab-7467-4e31-9f93-b2cd8fbb77ca admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:16.975507 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1346/2683] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:45:16.979622 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d70b0d-265d-4ee8-9f1b-7a85b87afefd admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:16.989723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d70b0d-265d-4ee8-9f1b-7a85b87afefd admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:16.994276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb72a8f-13f9-4ad1-bde2-bf5e6566254b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:16.994877 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb72a8f-13f9-4ad1-bde2-bf5e6566254b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:16.995199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb72a8f-13f9-4ad1-bde2-bf5e6566254b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:16.995482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb72a8f-13f9-4ad1-bde2-bf5e6566254b admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:16.996169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb72a8f-13f9-4ad1-bde2-bf5e6566254b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:16.997752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb72a8f-13f9-4ad1-bde2-bf5e6566254b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.009074 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1338/2684] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:17.013262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8ccab-7467-4e31-9f93-b2cd8fbb77ca admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:17.023732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8ccab-7467-4e31-9f93-b2cd8fbb77ca admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:17.028334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e67f96-f23e-4437-96c5-e8841f36f5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:17.028986 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e67f96-f23e-4437-96c5-e8841f36f5ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.029291 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e67f96-f23e-4437-96c5-e8841f36f5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.029556 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e67f96-f23e-4437-96c5-e8841f36f5ac admin admin] PATH_INFO: `/v3/projects/4b25c1ef0ac64c94bfa40d333cfa9f42/users/39ac6044e1644945ba76f3d9fa79c736/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.043700 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e67f96-f23e-4437-96c5-e8841f36f5ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b25c1ef0ac64c94bfa40d333cfa9f42, user_id=39ac6044e1644945ba76f3d9fa79c736, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:17.046351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e67f96-f23e-4437-96c5-e8841f36f5ac admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.059850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1347/2685] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:17 2022] PUT /identity/v3/projects/4b25c1ef0ac64c94bfa40d333cfa9f42/users/39ac6044e1644945ba76f3d9fa79c736/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:45:17.072015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34e02b-7708-471c-9a5d-1554bd1b256d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.072307 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34e02b-7708-471c-9a5d-1554bd1b256d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.072589 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34e02b-7708-471c-9a5d-1554bd1b256d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.100572 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6f34e02b-7708-471c-9a5d-1554bd1b256d None None] MFA Rules not processed for user `39ac6044e1644945ba76f3d9fa79c736`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:17.101721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6f34e02b-7708-471c-9a5d-1554bd1b256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:17.165358 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1339/2686] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:17.169991 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e67f96-f23e-4437-96c5-e8841f36f5ac admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:17.230652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e67f96-f23e-4437-96c5-e8841f36f5ac admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:17.235257 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08f2d-1eac-42da-b9fe-0a839944b46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:17.236065 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08f2d-1eac-42da-b9fe-0a839944b46f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.236351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08f2d-1eac-42da-b9fe-0a839944b46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.236631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08f2d-1eac-42da-b9fe-0a839944b46f admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.237295 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a08f2d-1eac-42da-b9fe-0a839944b46f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:17.239218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a08f2d-1eac-42da-b9fe-0a839944b46f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.270237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1348/2687] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:45:17 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:17.271468 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34e02b-7708-471c-9a5d-1554bd1b256d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:17.283095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34e02b-7708-471c-9a5d-1554bd1b256d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:17.341482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85894f61-d44a-41e9-a069-128ebaead75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:17.342096 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85894f61-d44a-41e9-a069-128ebaead75f admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.342377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85894f61-d44a-41e9-a069-128ebaead75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.342679 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85894f61-d44a-41e9-a069-128ebaead75f admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.343382 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85894f61-d44a-41e9-a069-128ebaead75f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:17.345376 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85894f61-d44a-41e9-a069-128ebaead75f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.365759 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1340/2688] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:45:17 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:17.369704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08f2d-1eac-42da-b9fe-0a839944b46f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:17.380811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08f2d-1eac-42da-b9fe-0a839944b46f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:17.385427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf27adc-b476-4302-a1ce-a5d0032fc842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:17.386023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf27adc-b476-4302-a1ce-a5d0032fc842 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.386308 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf27adc-b476-4302-a1ce-a5d0032fc842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.386584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf27adc-b476-4302-a1ce-a5d0032fc842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.387310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf27adc-b476-4302-a1ce-a5d0032fc842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:17.388887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf27adc-b476-4302-a1ce-a5d0032fc842 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.399869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1349/2689] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:17.403827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85894f61-d44a-41e9-a069-128ebaead75f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:17.413903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85894f61-d44a-41e9-a069-128ebaead75f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:17.418419 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adfd28-dde2-45cc-b2ca-89be095a9b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:17.419014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adfd28-dde2-45cc-b2ca-89be095a9b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.419328 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adfd28-dde2-45cc-b2ca-89be095a9b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.419606 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adfd28-dde2-45cc-b2ca-89be095a9b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.420290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89adfd28-dde2-45cc-b2ca-89be095a9b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:17.421903 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89adfd28-dde2-45cc-b2ca-89be095a9b42 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.432335 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1341/2690] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:17.436457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf27adc-b476-4302-a1ce-a5d0032fc842 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:17.446933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf27adc-b476-4302-a1ce-a5d0032fc842 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:17.451629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d13164-d838-4254-b0a1-bfac585c087d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:17.452266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d13164-d838-4254-b0a1-bfac585c087d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.452554 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d13164-d838-4254-b0a1-bfac585c087d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.452836 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d13164-d838-4254-b0a1-bfac585c087d admin admin] PATH_INFO: `/v3/projects/8a333b96c954432d9adb7b013fa6bf09/users/b41eb210307247318782b0fc572e996c/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.467344 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d13164-d838-4254-b0a1-bfac585c087d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a333b96c954432d9adb7b013fa6bf09, user_id=b41eb210307247318782b0fc572e996c, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:17.470072 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d13164-d838-4254-b0a1-bfac585c087d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.482589 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1350/2691] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:17 2022] PUT /identity/v3/projects/8a333b96c954432d9adb7b013fa6bf09/users/b41eb210307247318782b0fc572e996c/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:17.493254 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06551c94-6c46-4a6b-907e-b7e4db1a8b47 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.493539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06551c94-6c46-4a6b-907e-b7e4db1a8b47 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.493814 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06551c94-6c46-4a6b-907e-b7e4db1a8b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.521998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-06551c94-6c46-4a6b-907e-b7e4db1a8b47 None None] MFA Rules not processed for user `b41eb210307247318782b0fc572e996c`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:17.523233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-06551c94-6c46-4a6b-907e-b7e4db1a8b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:17.555582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:17.588131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1342/2692] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:17.592635 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d13164-d838-4254-b0a1-bfac585c087d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:17.654470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d13164-d838-4254-b0a1-bfac585c087d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:17.659167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10438635-cfba-41ea-8ce6-1063d52e4ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:17.659771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10438635-cfba-41ea-8ce6-1063d52e4ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.660058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10438635-cfba-41ea-8ce6-1063d52e4ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.660334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10438635-cfba-41ea-8ce6-1063d52e4ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.661001 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10438635-cfba-41ea-8ce6-1063d52e4ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:17.662882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10438635-cfba-41ea-8ce6-1063d52e4ca5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.683011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:17.691990 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1351/2693] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:45:17 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:17.696114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06551c94-6c46-4a6b-907e-b7e4db1a8b47 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:17.707907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06551c94-6c46-4a6b-907e-b7e4db1a8b47 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:17.764524 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dede16-59c7-48a2-8d7e-f3e774ba7201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:17.765179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dede16-59c7-48a2-8d7e-f3e774ba7201 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.765461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dede16-59c7-48a2-8d7e-f3e774ba7201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.765745 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dede16-59c7-48a2-8d7e-f3e774ba7201 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.766424 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dede16-59c7-48a2-8d7e-f3e774ba7201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:17.768508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dede16-59c7-48a2-8d7e-f3e774ba7201 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.789714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1343/2694] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:45:17 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:17.793830 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10438635-cfba-41ea-8ce6-1063d52e4ca5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:17.804927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10438635-cfba-41ea-8ce6-1063d52e4ca5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:17.809870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadbaa8-6e97-4add-a8d1-a8ba61f854be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:17.810549 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadbaa8-6e97-4add-a8d1-a8ba61f854be admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.810829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadbaa8-6e97-4add-a8d1-a8ba61f854be admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.811125 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadbaa8-6e97-4add-a8d1-a8ba61f854be admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.811815 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadbaa8-6e97-4add-a8d1-a8ba61f854be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:17.813426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadbaa8-6e97-4add-a8d1-a8ba61f854be admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.823863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1352/2695] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:17.828101 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dede16-59c7-48a2-8d7e-f3e774ba7201 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:17.838156 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dede16-59c7-48a2-8d7e-f3e774ba7201 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:17.842711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ab45c-d0c7-4244-95db-ae3d65da93eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:17.843402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ab45c-d0c7-4244-95db-ae3d65da93eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.843687 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ab45c-d0c7-4244-95db-ae3d65da93eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.843969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ab45c-d0c7-4244-95db-ae3d65da93eb admin admin] PATH_INFO: `/v3/projects/d3f60da3dc9d41169d8e656a6d4aea32/users/aef030a29c5848ff96f7e647f2e993b0/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.859160 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ab45c-d0c7-4244-95db-ae3d65da93eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f60da3dc9d41169d8e656a6d4aea32, user_id=aef030a29c5848ff96f7e647f2e993b0, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:17.862660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ab45c-d0c7-4244-95db-ae3d65da93eb admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.876323 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1344/2696] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:17 2022] PUT /identity/v3/projects/d3f60da3dc9d41169d8e656a6d4aea32/users/aef030a29c5848ff96f7e647f2e993b0/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:45:17.880240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadbaa8-6e97-4add-a8d1-a8ba61f854be admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:17.890089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadbaa8-6e97-4add-a8d1-a8ba61f854be admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:17.945111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bde55-544a-4171-a9a2-91f61c23ced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:17.945727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bde55-544a-4171-a9a2-91f61c23ced4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.946004 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bde55-544a-4171-a9a2-91f61c23ced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.946303 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bde55-544a-4171-a9a2-91f61c23ced4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.946966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bde55-544a-4171-a9a2-91f61c23ced4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:17.948594 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bde55-544a-4171-a9a2-91f61c23ced4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.959363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1353/2697] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:17.963525 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ab45c-d0c7-4244-95db-ae3d65da93eb admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:17.974133 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ab45c-d0c7-4244-95db-ae3d65da93eb admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:17.978866 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c28901-1a43-4ca8-ade2-a244cccab406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:17.979559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c28901-1a43-4ca8-ade2-a244cccab406 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:17.979850 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c28901-1a43-4ca8-ade2-a244cccab406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:17.980130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c28901-1a43-4ca8-ade2-a244cccab406 admin admin] PATH_INFO: `/v3/projects/d3f60da3dc9d41169d8e656a6d4aea32/users/aef030a29c5848ff96f7e647f2e993b0/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:17.982150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c28901-1a43-4ca8-ade2-a244cccab406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f60da3dc9d41169d8e656a6d4aea32, user_id=aef030a29c5848ff96f7e647f2e993b0, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:17.984841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c28901-1a43-4ca8-ade2-a244cccab406 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:17.996856 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1345/2698] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:17 2022] PUT /identity/v3/projects/d3f60da3dc9d41169d8e656a6d4aea32/users/aef030a29c5848ff96f7e647f2e993b0/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:18.008067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910571e-689c-4eb0-898b-2c16996b699f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:18.008398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910571e-689c-4eb0-898b-2c16996b699f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:18.008689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910571e-689c-4eb0-898b-2c16996b699f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:18.036411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c910571e-689c-4eb0-898b-2c16996b699f None None] MFA Rules not processed for user `aef030a29c5848ff96f7e647f2e993b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:18.037623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c910571e-689c-4eb0-898b-2c16996b699f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:18.103925 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1354/2699] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:18.115055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c28901-1a43-4ca8-ade2-a244cccab406 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:18.116793 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-70c28901-1a43-4ca8-ade2-a244cccab406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:18.176247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c28901-1a43-4ca8-ade2-a244cccab406 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:18.179885 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc4008-f537-4e3a-81c9-ec88133b1d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:18.180489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc4008-f537-4e3a-81c9-ec88133b1d15 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:18.180771 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc4008-f537-4e3a-81c9-ec88133b1d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:18.181052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc4008-f537-4e3a-81c9-ec88133b1d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:18.224518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:18.240540 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccc4008-f537-4e3a-81c9-ec88133b1d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:18.242002 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccc4008-f537-4e3a-81c9-ec88133b1d15 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:18.251539 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1346/2700] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:19.207504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910571e-689c-4eb0-898b-2c16996b699f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:19.217028 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910571e-689c-4eb0-898b-2c16996b699f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:19.221415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed032293-c23a-4920-adfb-bd3b62273113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:19.222011 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed032293-c23a-4920-adfb-bd3b62273113 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:19.222292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed032293-c23a-4920-adfb-bd3b62273113 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:19.222567 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed032293-c23a-4920-adfb-bd3b62273113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:19.224296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ed032293-c23a-4920-adfb-bd3b62273113 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:19.284222 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed032293-c23a-4920-adfb-bd3b62273113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:19.285661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed032293-c23a-4920-adfb-bd3b62273113 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:19.294721 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1355/2701] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:19.750292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc4008-f537-4e3a-81c9-ec88133b1d15 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:19.752511 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-cccc4008-f537-4e3a-81c9-ec88133b1d15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:19.763911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc4008-f537-4e3a-81c9-ec88133b1d15 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:19.769492 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76db4e-571e-40a5-893c-95c517569987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:19.771800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76db4e-571e-40a5-893c-95c517569987 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:19.772494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76db4e-571e-40a5-893c-95c517569987 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:19.773004 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76db4e-571e-40a5-893c-95c517569987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:19.834941 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76db4e-571e-40a5-893c-95c517569987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:19.836421 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76db4e-571e-40a5-893c-95c517569987 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:19.845744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1347/2702] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:20.840869 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed032293-c23a-4920-adfb-bd3b62273113 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:20.849585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed032293-c23a-4920-adfb-bd3b62273113 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:20.853213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac07938-4e21-44b7-9fa5-da861c064f02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:20.853814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac07938-4e21-44b7-9fa5-da861c064f02 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:20.854099 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac07938-4e21-44b7-9fa5-da861c064f02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:20.854397 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac07938-4e21-44b7-9fa5-da861c064f02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:20.860199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:20.864460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac07938-4e21-44b7-9fa5-da861c064f02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:20.866039 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac07938-4e21-44b7-9fa5-da861c064f02 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:20.876452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1356/2703] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:20 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:25.088938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4836f0-cf5c-4bfd-a670-fe94544820bf None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:25.089423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4836f0-cf5c-4bfd-a670-fe94544820bf None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:25.089551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4836f0-cf5c-4bfd-a670-fe94544820bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:25.108537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-9f4836f0-cf5c-4bfd-a670-fe94544820bf None None] MFA Rules not processed for user `1ba46a929ed54c51a33662367289eee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:25.109749 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9f4836f0-cf5c-4bfd-a670-fe94544820bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:25.163714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1348/2704] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:26.956220 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac07938-4e21-44b7-9fa5-da861c064f02 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:27.019986 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac07938-4e21-44b7-9fa5-da861c064f02 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:27.024745 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:27.025383 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:27.025665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:27.025947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] PATH_INFO: `/v3/users/39ac6044e1644945ba76f3d9fa79c736` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:27.027128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=39ac6044e1644945ba76f3d9fa79c736)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:27.029045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:27.074673 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39ac6044e1644945ba76f3d9fa79c736'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.083528 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39ac6044e1644945ba76f3d9fa79c736'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.096406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39ac6044e1644945ba76f3d9fa79c736'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.097076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39ac6044e1644945ba76f3d9fa79c736'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.097693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39ac6044e1644945ba76f3d9fa79c736'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.109897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1357/2705] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:26 2022] DELETE /identity/v3/users/39ac6044e1644945ba76f3d9fa79c736 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:27.117230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e11fb7-216a-4652-9521-85a79a9e87ea None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:27.117230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e11fb7-216a-4652-9521-85a79a9e87ea None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:27.117230 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e11fb7-216a-4652-9521-85a79a9e87ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:27.136742 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-77e11fb7-216a-4652-9521-85a79a9e87ea None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:27.137964 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-77e11fb7-216a-4652-9521-85a79a9e87ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:27.212389 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1349/2706] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:45:27.222462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:27.225648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:27.324009 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92c183-2a1f-4289-8719-5aa46db669ae admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:27.328218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d567d-1dcc-4570-badb-9ac4bca73c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:27.328839 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d567d-1dcc-4570-badb-9ac4bca73c16 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:27.329131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d567d-1dcc-4570-badb-9ac4bca73c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:27.329524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d567d-1dcc-4570-badb-9ac4bca73c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:27.338654 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:27.341752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d567d-1dcc-4570-badb-9ac4bca73c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:27.341911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e36543-4c6b-475b-9731-35a6af9bd2df None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:27.342340 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e36543-4c6b-475b-9731-35a6af9bd2df None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:27.342777 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e36543-4c6b-475b-9731-35a6af9bd2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:27.343249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d567d-1dcc-4570-badb-9ac4bca73c16 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:27.356043 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1358/2707] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:27.382522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a7e36543-4c6b-475b-9731-35a6af9bd2df None None] MFA Rules not processed for user `890be2e19b43473caa79b2dc889f8436`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:27.383906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a7e36543-4c6b-475b-9731-35a6af9bd2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:27.415363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:27.474711 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1350/2708] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:27.696648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d567d-1dcc-4570-badb-9ac4bca73c16 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:27.698130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f13d567d-1dcc-4570-badb-9ac4bca73c16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:27.715001 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d567d-1dcc-4570-badb-9ac4bca73c16 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:27.721042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:27.721834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:27.722558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:27.722911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] PATH_INFO: `/v3/projects/4b25c1ef0ac64c94bfa40d333cfa9f42` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:27.724218 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b25c1ef0ac64c94bfa40d333cfa9f42)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:27.726234 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:27.785688 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b25c1ef0ac64c94bfa40d333cfa9f42', 'request_id': 'req-f1bb3651-27c9-4cfb-8809-b24814564ed9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.793249 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b25c1ef0ac64c94bfa40d333cfa9f42', 'request_id': 'req-f1bb3651-27c9-4cfb-8809-b24814564ed9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.799433 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b25c1ef0ac64c94bfa40d333cfa9f42', 'request_id': 'req-f1bb3651-27c9-4cfb-8809-b24814564ed9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.819425 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] The token cache is being invalidate because project 4b25c1ef0ac64c94bfa40d333cfa9f42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:27.819717 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.820385 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.822023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1359/2709] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:27 2022] DELETE /identity/v3/projects/4b25c1ef0ac64c94bfa40d333cfa9f42 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:27.825993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e36543-4c6b-475b-9731-35a6af9bd2df None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:27.838245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e36543-4c6b-475b-9731-35a6af9bd2df None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:27.895884 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:27.896437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:27.896716 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:27.896990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] PATH_INFO: `/v3/users/b41eb210307247318782b0fc572e996c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:27.898193 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41eb210307247318782b0fc572e996c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:27.900141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:27.953717 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41eb210307247318782b0fc572e996c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.977190 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41eb210307247318782b0fc572e996c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.978050 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41eb210307247318782b0fc572e996c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:27.988177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41eb210307247318782b0fc572e996c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:28.005040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41eb210307247318782b0fc572e996c'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:28.013342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1351/2710] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:27 2022] DELETE /identity/v3/users/b41eb210307247318782b0fc572e996c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:28.404005 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:28.406351 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:28.427354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb3651-27c9-4cfb-8809-b24814564ed9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:28.535452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:28.536140 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:28.536429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:28.536705 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] PATH_INFO: `/v3/projects/8a333b96c954432d9adb7b013fa6bf09` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:28.537919 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a333b96c954432d9adb7b013fa6bf09)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:28.540353 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:28.613498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a333b96c954432d9adb7b013fa6bf09', 'request_id': 'req-bf3547d5-3b17-4188-b92b-f0250305f16a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:28.621741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a333b96c954432d9adb7b013fa6bf09', 'request_id': 'req-bf3547d5-3b17-4188-b92b-f0250305f16a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:28.634211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a333b96c954432d9adb7b013fa6bf09', 'request_id': 'req-bf3547d5-3b17-4188-b92b-f0250305f16a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:28.646606 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] The token cache is being invalidate because project 8a333b96c954432d9adb7b013fa6bf09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:28.646895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:28.647793 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:28.649563 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1360/2711] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:28 2022] DELETE /identity/v3/projects/8a333b96c954432d9adb7b013fa6bf09 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:28.653291 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:28.655266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:28.668276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a903357-ec2f-4d5c-8efb-671394f867a9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:28.721910 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:28.744522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e202c-c6e8-4e4d-9414-322bef9bb86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:28.745239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e202c-c6e8-4e4d-9414-322bef9bb86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:28.745518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e202c-c6e8-4e4d-9414-322bef9bb86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:28.745976 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e202c-c6e8-4e4d-9414-322bef9bb86c admin admin] PATH_INFO: `/v3/users/aef030a29c5848ff96f7e647f2e993b0` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:28.747369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e202c-c6e8-4e4d-9414-322bef9bb86c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef030a29c5848ff96f7e647f2e993b0)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:28.749938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e202c-c6e8-4e4d-9414-322bef9bb86c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:28.811325 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e31e202c-c6e8-4e4d-9414-322bef9bb86c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef030a29c5848ff96f7e647f2e993b0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:28.826171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e31e202c-c6e8-4e4d-9414-322bef9bb86c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef030a29c5848ff96f7e647f2e993b0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:28.827043 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e31e202c-c6e8-4e4d-9414-322bef9bb86c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef030a29c5848ff96f7e647f2e993b0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:28.840195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e31e202c-c6e8-4e4d-9414-322bef9bb86c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef030a29c5848ff96f7e647f2e993b0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:28.849802 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-e31e202c-c6e8-4e4d-9414-322bef9bb86c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef030a29c5848ff96f7e647f2e993b0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:28.857744 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1352/2712] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:28 2022] DELETE /identity/v3/users/aef030a29c5848ff96f7e647f2e993b0 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:45:29.237061 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:29.239384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:29.258277 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3547d5-3b17-4188-b92b-f0250305f16a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:29.264128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:29.320803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b6cef-f914-4b97-b9c3-2f8e788b960f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:29.321460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b6cef-f914-4b97-b9c3-2f8e788b960f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:29.321748 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b6cef-f914-4b97-b9c3-2f8e788b960f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:29.322028 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b6cef-f914-4b97-b9c3-2f8e788b960f admin admin] PATH_INFO: `/v3/projects/d3f60da3dc9d41169d8e656a6d4aea32` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:29.323240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b6cef-f914-4b97-b9c3-2f8e788b960f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f60da3dc9d41169d8e656a6d4aea32)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:29.325136 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b6cef-f914-4b97-b9c3-2f8e788b960f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:29.371637 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f37b6cef-f914-4b97-b9c3-2f8e788b960f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f60da3dc9d41169d8e656a6d4aea32', 'request_id': 'req-f37b6cef-f914-4b97-b9c3-2f8e788b960f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:29.380271 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f37b6cef-f914-4b97-b9c3-2f8e788b960f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f60da3dc9d41169d8e656a6d4aea32', 'request_id': 'req-f37b6cef-f914-4b97-b9c3-2f8e788b960f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:29.385874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f37b6cef-f914-4b97-b9c3-2f8e788b960f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f60da3dc9d41169d8e656a6d4aea32', 'request_id': 'req-f37b6cef-f914-4b97-b9c3-2f8e788b960f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:29.397296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f37b6cef-f914-4b97-b9c3-2f8e788b960f admin admin] The token cache is being invalidate because project d3f60da3dc9d41169d8e656a6d4aea32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:29.397583 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f37b6cef-f914-4b97-b9c3-2f8e788b960f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:29.398213 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f37b6cef-f914-4b97-b9c3-2f8e788b960f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:29.399738 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1361/2713] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:29 2022] DELETE /identity/v3/projects/d3f60da3dc9d41169d8e656a6d4aea32 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:45:29.404429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d4526-f132-4c14-86e1-4d5b8017094d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:29.404719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d4526-f132-4c14-86e1-4d5b8017094d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:29.405000 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d4526-f132-4c14-86e1-4d5b8017094d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:29.425625 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e69d4526-f132-4c14-86e1-4d5b8017094d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:29.426819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e69d4526-f132-4c14-86e1-4d5b8017094d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:29.484816 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1353/2714] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:29.493312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ce737-76c7-4667-a1c9-b3a5705b60ba None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:29.493774 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ce737-76c7-4667-a1c9-b3a5705b60ba None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:29.493926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ce737-76c7-4667-a1c9-b3a5705b60ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:29.513629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-642ce737-76c7-4667-a1c9-b3a5705b60ba None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:29.514826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-642ce737-76c7-4667-a1c9-b3a5705b60ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:29.523741 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1362/2715] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:29.527861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d4526-f132-4c14-86e1-4d5b8017094d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:29.538609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d4526-f132-4c14-86e1-4d5b8017094d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:29.543187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6b1c4-3887-4e12-a874-31c71da5be08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:29.543799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6b1c4-3887-4e12-a874-31c71da5be08 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:29.544109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6b1c4-3887-4e12-a874-31c71da5be08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:29.544493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6b1c4-3887-4e12-a874-31c71da5be08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:29.545188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6b1c4-3887-4e12-a874-31c71da5be08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:45:29.545493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6b1c4-3887-4e12-a874-31c71da5be08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:29.547109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6b1c4-3887-4e12-a874-31c71da5be08 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:29.561059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1354/2716] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:45:29.565681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ce737-76c7-4667-a1c9-b3a5705b60ba None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:29.574968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ce737-76c7-4667-a1c9-b3a5705b60ba None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:29.580616 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019c133-771e-4562-8da9-c378e09e9bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:29.581217 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019c133-771e-4562-8da9-c378e09e9bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:29.581504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019c133-771e-4562-8da9-c378e09e9bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:29.581786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019c133-771e-4562-8da9-c378e09e9bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:29.582456 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019c133-771e-4562-8da9-c378e09e9bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:29.584506 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019c133-771e-4562-8da9-c378e09e9bc4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:29.612263 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1363/2717] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:45:29 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:29.616163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6b1c4-3887-4e12-a874-31c71da5be08 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:29.626239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6b1c4-3887-4e12-a874-31c71da5be08 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:29.675563 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:29.682177 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7667a3d-fff8-40dd-bf6a-51372bf1e1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:29.682956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7667a3d-fff8-40dd-bf6a-51372bf1e1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:29.683451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7667a3d-fff8-40dd-bf6a-51372bf1e1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:29.684258 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7667a3d-fff8-40dd-bf6a-51372bf1e1a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:29.685145 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7667a3d-fff8-40dd-bf6a-51372bf1e1a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:29.687219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7667a3d-fff8-40dd-bf6a-51372bf1e1a7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:29.708310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1355/2718] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:45:29 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:45:29.712345 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019c133-771e-4562-8da9-c378e09e9bc4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:29.723223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019c133-771e-4562-8da9-c378e09e9bc4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:29.727987 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a651494-89e2-4817-a25a-2e0d542c965a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:29.728594 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a651494-89e2-4817-a25a-2e0d542c965a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:29.728880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a651494-89e2-4817-a25a-2e0d542c965a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:29.729159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a651494-89e2-4817-a25a-2e0d542c965a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:29.729883 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a651494-89e2-4817-a25a-2e0d542c965a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:29.731515 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a651494-89e2-4817-a25a-2e0d542c965a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:29.741655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1364/2719] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:29.745390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7667a3d-fff8-40dd-bf6a-51372bf1e1a7 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:29.756130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7667a3d-fff8-40dd-bf6a-51372bf1e1a7 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:29.760630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035ee4f-eb22-4bde-ab9a-38a134c1a41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:29.761225 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035ee4f-eb22-4bde-ab9a-38a134c1a41a admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:29.761513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035ee4f-eb22-4bde-ab9a-38a134c1a41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:29.761789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035ee4f-eb22-4bde-ab9a-38a134c1a41a admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:29.762469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035ee4f-eb22-4bde-ab9a-38a134c1a41a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:29.764084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035ee4f-eb22-4bde-ab9a-38a134c1a41a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:29.776046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1356/2720] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:45:29.780200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a651494-89e2-4817-a25a-2e0d542c965a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:29.791882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a651494-89e2-4817-a25a-2e0d542c965a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:29.796786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64a903-466d-4880-841b-5ab36c82b3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:29.797652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64a903-466d-4880-841b-5ab36c82b3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:29.798194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64a903-466d-4880-841b-5ab36c82b3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:29.798819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64a903-466d-4880-841b-5ab36c82b3ea admin admin] PATH_INFO: `/v3/projects/8849e0283d3543e380f3cf568cb1be2a/users/f9ef8ca60477408dbe7433c31a7f1a50/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:29.817882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64a903-466d-4880-841b-5ab36c82b3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8849e0283d3543e380f3cf568cb1be2a, user_id=f9ef8ca60477408dbe7433c31a7f1a50, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:29.821003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64a903-466d-4880-841b-5ab36c82b3ea admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:29.836477 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1365/2721] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:45:29 2022] PUT /identity/v3/projects/8849e0283d3543e380f3cf568cb1be2a/users/f9ef8ca60477408dbe7433c31a7f1a50/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:45:29.846895 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa8bbc-ed6f-4070-8417-9930e72ce563 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:29.847210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa8bbc-ed6f-4070-8417-9930e72ce563 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:29.847494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa8bbc-ed6f-4070-8417-9930e72ce563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:29.875926 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-88aa8bbc-ed6f-4070-8417-9930e72ce563 None None] MFA Rules not processed for user `f9ef8ca60477408dbe7433c31a7f1a50`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:29.877956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-88aa8bbc-ed6f-4070-8417-9930e72ce563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:29.941514 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1357/2722] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:29.952327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64a903-466d-4880-841b-5ab36c82b3ea admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:29.953984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2a64a903-466d-4880-841b-5ab36c82b3ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:30.000964 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:30.026666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64a903-466d-4880-841b-5ab36c82b3ea admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:30.033007 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0d55c-cff2-4455-8b59-621c5cbfb3df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:30.033684 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0d55c-cff2-4455-8b59-621c5cbfb3df service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:30.033973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0d55c-cff2-4455-8b59-621c5cbfb3df service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:30.034248 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0d55c-cff2-4455-8b59-621c5cbfb3df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:30.044897 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0d55c-cff2-4455-8b59-621c5cbfb3df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:30.046411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0d55c-cff2-4455-8b59-621c5cbfb3df service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:30.058010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1366/2723] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:31.464675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b0765-6f90-4344-8b37-45a73b5faccd None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:31.465245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b0765-6f90-4344-8b37-45a73b5faccd None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:31.465245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b0765-6f90-4344-8b37-45a73b5faccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:31.485094 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-3c9b0765-6f90-4344-8b37-45a73b5faccd None None] MFA Rules not processed for user `f9ef8ca60477408dbe7433c31a7f1a50`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:31.486398 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3c9b0765-6f90-4344-8b37-45a73b5faccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:31.494513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1358/2724] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:40.052939 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=10 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:41.480061 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=43,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:48.489725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a80867-c5fd-4b66-b0ff-7e4aeb19e347 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:48.490231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a80867-c5fd-4b66-b0ff-7e4aeb19e347 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:48.490342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a80867-c5fd-4b66-b0ff-7e4aeb19e347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:48.520286 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d7a80867-c5fd-4b66-b0ff-7e4aeb19e347 None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:48.521491 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d7a80867-c5fd-4b66-b0ff-7e4aeb19e347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:48.585759 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1367/2725] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:45:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:48.636134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b0765-6f90-4344-8b37-45a73b5faccd None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:48.637592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3c9b0765-6f90-4344-8b37-45a73b5faccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:48.712658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b0765-6f90-4344-8b37-45a73b5faccd None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:48.717760 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc7ac-3aa2-40f2-86d8-e413bbb13da5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:48.718399 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc7ac-3aa2-40f2-86d8-e413bbb13da5 service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:48.718687 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc7ac-3aa2-40f2-86d8-e413bbb13da5 service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:48.718952 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc7ac-3aa2-40f2-86d8-e413bbb13da5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:48.728395 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdc7ac-3aa2-40f2-86d8-e413bbb13da5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:48.729920 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdc7ac-3aa2-40f2-86d8-e413bbb13da5 service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:48.739116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1359/2726] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:45:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:45:49.261895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dc807-da40-4f3c-bf7a-2cc081b9406c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:49.262285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dc807-da40-4f3c-bf7a-2cc081b9406c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:49.262638 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dc807-da40-4f3c-bf7a-2cc081b9406c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:49.284092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0b2dc807-da40-4f3c-bf7a-2cc081b9406c None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:49.285752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0b2dc807-da40-4f3c-bf7a-2cc081b9406c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:49.295545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1368/2727] 10.222.0.40 () {58 vars in 967 bytes} [Mon Aug 1 07:45:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:49.309040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc7ac-3aa2-40f2-86d8-e413bbb13da5 service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:49.320233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc7ac-3aa2-40f2-86d8-e413bbb13da5 service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:49.325409 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88332720-d506-488f-99c5-cf1b74968f75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:49.326013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88332720-d506-488f-99c5-cf1b74968f75 service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:49.326301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88332720-d506-488f-99c5-cf1b74968f75 service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:49.326578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88332720-d506-488f-99c5-cf1b74968f75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:49.335466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88332720-d506-488f-99c5-cf1b74968f75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:49.337645 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88332720-d506-488f-99c5-cf1b74968f75 service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:49.346780 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1360/2728] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:45:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:45:49.566414 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818087b-c09d-4e95-ae37-2d300d1ed6ca None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:49.566888 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818087b-c09d-4e95-ae37-2d300d1ed6ca None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:49.567000 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818087b-c09d-4e95-ae37-2d300d1ed6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:49.585927 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0818087b-c09d-4e95-ae37-2d300d1ed6ca None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:49.587167 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0818087b-c09d-4e95-ae37-2d300d1ed6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:49.594980 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1369/2729] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Aug 1 07:45:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:49.607089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88332720-d506-488f-99c5-cf1b74968f75 service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:49.619373 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88332720-d506-488f-99c5-cf1b74968f75 service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:49.624724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f2a6e-dabf-400e-b93c-78fe7ef43e26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:49.625335 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f2a6e-dabf-400e-b93c-78fe7ef43e26 service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:49.625678 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f2a6e-dabf-400e-b93c-78fe7ef43e26 service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:49.625967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f2a6e-dabf-400e-b93c-78fe7ef43e26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:49.634373 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f2a6e-dabf-400e-b93c-78fe7ef43e26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:49.635974 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f2a6e-dabf-400e-b93c-78fe7ef43e26 service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:49.644946 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1361/2730] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:45:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:45:50.004508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398eb1c-5805-4ebb-b774-057c5de85f18 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:50.004863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398eb1c-5805-4ebb-b774-057c5de85f18 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:50.005746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398eb1c-5805-4ebb-b774-057c5de85f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:50.024971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a398eb1c-5805-4ebb-b774-057c5de85f18 None None] MFA Rules not processed for user `8340fafe48af45c8b87a97aaac637bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:50.026312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a398eb1c-5805-4ebb-b774-057c5de85f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:50.034298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1370/2731] 10.222.0.40 () {58 vars in 967 bytes} [Mon Aug 1 07:45:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:45:50.051031 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f2a6e-dabf-400e-b93c-78fe7ef43e26 service swift] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:50.061999 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f2a6e-dabf-400e-b93c-78fe7ef43e26 service swift] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:50.067863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866fcc3-7ac5-47c5-a331-9edfcfe4b1de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ba2655c2eef408498c0298d432263f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:50.068570 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866fcc3-7ac5-47c5-a331-9edfcfe4b1de service swift] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:50.068918 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866fcc3-7ac5-47c5-a331-9edfcfe4b1de service swift] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:50.069260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866fcc3-7ac5-47c5-a331-9edfcfe4b1de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:50.078362 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866fcc3-7ac5-47c5-a331-9edfcfe4b1de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:50.080305 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866fcc3-7ac5-47c5-a331-9edfcfe4b1de service swift] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:50.089157 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1362/2732] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Aug 1 07:45:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 07:45:52.247243 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398eb1c-5805-4ebb-b774-057c5de85f18 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:52.309402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398eb1c-5805-4ebb-b774-057c5de85f18 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:52.314006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:52.314691 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:52.315622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:52.315622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] PATH_INFO: `/v3/users/f9ef8ca60477408dbe7433c31a7f1a50` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:52.317854 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9ef8ca60477408dbe7433c31a7f1a50)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:52.319797 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:52.365229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9ef8ca60477408dbe7433c31a7f1a50'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:52.372545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9ef8ca60477408dbe7433c31a7f1a50'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:52.384920 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9ef8ca60477408dbe7433c31a7f1a50'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:52.385550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9ef8ca60477408dbe7433c31a7f1a50'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:52.386172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9ef8ca60477408dbe7433c31a7f1a50'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:52.387382 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:52.401922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1371/2733] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:45:52 2022] DELETE /identity/v3/users/f9ef8ca60477408dbe7433c31a7f1a50 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:45:52.406385 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897f8ef-3e25-4023-84a3-b7723ab9b3c6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:52.407456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897f8ef-3e25-4023-84a3-b7723ab9b3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:52.407624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897f8ef-3e25-4023-84a3-b7723ab9b3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:52.426874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8897f8ef-3e25-4023-84a3-b7723ab9b3c6 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:45:52.428130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8897f8ef-3e25-4023-84a3-b7723ab9b3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:52.486452 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1363/2734] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:45:52.496159 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:52.497570 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:52.554776 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee62da-d9d6-4af1-b7e2-825bfcd5a07b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:52.558390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413584c8-4066-4a84-84b5-e285864e7778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:52.558996 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413584c8-4066-4a84-84b5-e285864e7778 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:52.559324 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413584c8-4066-4a84-84b5-e285864e7778 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:52.559605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413584c8-4066-4a84-84b5-e285864e7778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:52.568767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413584c8-4066-4a84-84b5-e285864e7778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:52.570200 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413584c8-4066-4a84-84b5-e285864e7778 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:52.579614 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1372/2735] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:45:52.751155 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897f8ef-3e25-4023-84a3-b7723ab9b3c6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:45:52.752632 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8897f8ef-3e25-4023-84a3-b7723ab9b3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:45:52.768449 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897f8ef-3e25-4023-84a3-b7723ab9b3c6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:45:52.773907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:45:52.774508 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:45:52.774787 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:45:52.775084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] PATH_INFO: `/v3/projects/8849e0283d3543e380f3cf568cb1be2a` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:45:52.776326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=8849e0283d3543e380f3cf568cb1be2a)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:45:52.778377 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:45:52.823264 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:45:52.826501 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8849e0283d3543e380f3cf568cb1be2a', 'request_id': 'req-039d4241-0078-4237-9bf1-066a29cd9aee'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:52.833704 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8849e0283d3543e380f3cf568cb1be2a', 'request_id': 'req-039d4241-0078-4237-9bf1-066a29cd9aee'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:52.839795 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8849e0283d3543e380f3cf568cb1be2a', 'request_id': 'req-039d4241-0078-4237-9bf1-066a29cd9aee'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:52.852293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] The token cache is being invalidate because project 8849e0283d3543e380f3cf568cb1be2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:45:52.852578 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:52.853196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:45:52.854675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1364/2736] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:45:52 2022] DELETE /identity/v3/projects/8849e0283d3543e380f3cf568cb1be2a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:46:02.575538 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=31 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:46:02.849411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:46:26.237417 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265dce94-0329-4d4c-bfb8-aaec3ab2efbd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:46:26.237917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265dce94-0329-4d4c-bfb8-aaec3ab2efbd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:46:26.238036 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265dce94-0329-4d4c-bfb8-aaec3ab2efbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:46:26.257246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-265dce94-0329-4d4c-bfb8-aaec3ab2efbd None None] MFA Rules not processed for user `1ba46a929ed54c51a33662367289eee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:46:26.258450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-265dce94-0329-4d4c-bfb8-aaec3ab2efbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:46:26.313271 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1373/2737] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:46:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:46:36.306082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=25 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:47:16.276779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:47:16.278208 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:47:16.358519 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d4241-0078-4237-9bf1-066a29cd9aee admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:47:16.362082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1e368-f5c4-4bc5-9aac-86ec0f662c57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:47:16.362812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1e368-f5c4-4bc5-9aac-86ec0f662c57 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:47:16.363116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1e368-f5c4-4bc5-9aac-86ec0f662c57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:47:16.363391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1e368-f5c4-4bc5-9aac-86ec0f662c57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:47:16.365111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-3fe1e368-f5c4-4bc5-9aac-86ec0f662c57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:47:16.445616 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1e368-f5c4-4bc5-9aac-86ec0f662c57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:47:16.446956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1e368-f5c4-4bc5-9aac-86ec0f662c57 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:47:16.466823 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1365/2738] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:47:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 07:47:24.241450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265dce94-0329-4d4c-bfb8-aaec3ab2efbd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:47:24.251469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265dce94-0329-4d4c-bfb8-aaec3ab2efbd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:47:24.255819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019d935-b879-4865-ba72-30d5d267f294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:47:24.256455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019d935-b879-4865-ba72-30d5d267f294 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:47:24.256735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019d935-b879-4865-ba72-30d5d267f294 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:47:24.257010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019d935-b879-4865-ba72-30d5d267f294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:47:24.258725 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b019d935-b879-4865-ba72-30d5d267f294 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:47:24.271777 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019d935-b879-4865-ba72-30d5d267f294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:47:24.273182 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019d935-b879-4865-ba72-30d5d267f294 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:47:24.283787 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1374/2739] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:47:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:47:26.464256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=72 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:47:32.362615 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8362c74-0878-43bb-9dbd-b366e38de075 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:47:32.363266 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8362c74-0878-43bb-9dbd-b366e38de075 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:47:32.363384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8362c74-0878-43bb-9dbd-b366e38de075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:47:32.387732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c8362c74-0878-43bb-9dbd-b366e38de075 None None] MFA Rules not processed for user `1ba46a929ed54c51a33662367289eee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:47:32.389043 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c8362c74-0878-43bb-9dbd-b366e38de075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:47:32.398030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1366/2740] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:47:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:47:34.279093 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:47:42.381546 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:47:42.410995 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019d935-b879-4865-ba72-30d5d267f294 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:47:42.412604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b019d935-b879-4865-ba72-30d5d267f294 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:47:42.468847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019d935-b879-4865-ba72-30d5d267f294 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:47:42.472969 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91324f-2c9b-42da-90e8-42e225cdb92e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:47:42.474023 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91324f-2c9b-42da-90e8-42e225cdb92e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:47:42.474525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91324f-2c9b-42da-90e8-42e225cdb92e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:47:42.475027 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91324f-2c9b-42da-90e8-42e225cdb92e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:47:42.477302 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4c91324f-2c9b-42da-90e8-42e225cdb92e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:47:42.541222 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91324f-2c9b-42da-90e8-42e225cdb92e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:47:42.542650 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91324f-2c9b-42da-90e8-42e225cdb92e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:47:42.552783 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1375/2741] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:47:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:47:49.338655 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8362c74-0878-43bb-9dbd-b366e38de075 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:47:49.340210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c8362c74-0878-43bb-9dbd-b366e38de075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:47:49.354522 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8362c74-0878-43bb-9dbd-b366e38de075 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:47:49.358944 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:47:49.359606 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:47:49.359888 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:47:49.360162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] PATH_INFO: `/v3/users/890be2e19b43473caa79b2dc889f8436` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:47:49.361319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] RBAC: Authorizing `identity:delete_user(user_id=890be2e19b43473caa79b2dc889f8436)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:47:49.363285 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:47:49.408909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '890be2e19b43473caa79b2dc889f8436'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.421841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '890be2e19b43473caa79b2dc889f8436'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.422555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '890be2e19b43473caa79b2dc889f8436'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.430185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '890be2e19b43473caa79b2dc889f8436'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.437583 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '890be2e19b43473caa79b2dc889f8436'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.444390 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1367/2742] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:47:49 2022] DELETE /identity/v3/users/890be2e19b43473caa79b2dc889f8436 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:47:49.622490 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91324f-2c9b-42da-90e8-42e225cdb92e service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:47:49.634469 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91324f-2c9b-42da-90e8-42e225cdb92e service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:47:49.692161 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:47:49.692799 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:47:49.693087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:47:49.693369 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] PATH_INFO: `/v3/projects/810069989cfb42799139be8ec2423288` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:47:49.694525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=810069989cfb42799139be8ec2423288)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:47:49.696453 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:47:49.742164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '810069989cfb42799139be8ec2423288', 'request_id': 'req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.744656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:47:49.749898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '810069989cfb42799139be8ec2423288', 'request_id': 'req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.756895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '810069989cfb42799139be8ec2423288', 'request_id': 'req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.767641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] The token cache is being invalidate because project 810069989cfb42799139be8ec2423288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:47:49.767932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.769341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.771241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1376/2743] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:47:49 2022] DELETE /identity/v3/projects/810069989cfb42799139be8ec2423288 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:47:49.775292 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:47:49.776774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:47:49.790874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf68b7a-d968-4c15-89d5-b1b87030250b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:47:49.848247 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:47:49.848886 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:47:49.849171 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:47:49.849450 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] PATH_INFO: `/v3/users/5d7cf9c678b44bc9876e2c1efaa7277d` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:47:49.850637 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d7cf9c678b44bc9876e2c1efaa7277d)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:47:49.852582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:47:49.907393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d7cf9c678b44bc9876e2c1efaa7277d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.920343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d7cf9c678b44bc9876e2c1efaa7277d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.921048 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d7cf9c678b44bc9876e2c1efaa7277d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.928477 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d7cf9c678b44bc9876e2c1efaa7277d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.930260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:47:49.937841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d7cf9c678b44bc9876e2c1efaa7277d'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:49.949210 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1368/2744] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:47:49 2022] DELETE /identity/v3/users/5d7cf9c678b44bc9876e2c1efaa7277d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:47:50.133767 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:47:50.135251 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:47:50.148749 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62a1a6-9b35-4bfd-afc1-5bfcaaac7a85 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:47:50.204665 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:47:50.205310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:47:50.205593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:47:50.205870 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] PATH_INFO: `/v3/projects/ec8569655b2542939019e71ea94d1ddd` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:47:50.207823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8569655b2542939019e71ea94d1ddd)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:47:50.209760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:47:50.254851 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8569655b2542939019e71ea94d1ddd', 'request_id': 'req-31879030-7247-4c2e-9563-1fd422db7ed6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:50.261495 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8569655b2542939019e71ea94d1ddd', 'request_id': 'req-31879030-7247-4c2e-9563-1fd422db7ed6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:50.267010 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8569655b2542939019e71ea94d1ddd', 'request_id': 'req-31879030-7247-4c2e-9563-1fd422db7ed6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:50.278492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] The token cache is being invalidate because project ec8569655b2542939019e71ea94d1ddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:47:50.278782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:50.279438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:47:50.280926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1377/2745] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:47:50 2022] DELETE /identity/v3/projects/ec8569655b2542939019e71ea94d1ddd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:47:59.954730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:48:00.275584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:48:22.694980 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:22.697188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:22.754054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975f489-b599-4c69-9f36-bf0bd92dd6b9 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:22.801627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9a7ba-094f-485e-9b16-82504ad2a4d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:22.802245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9a7ba-094f-485e-9b16-82504ad2a4d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:22.802531 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9a7ba-094f-485e-9b16-82504ad2a4d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:22.802813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9a7ba-094f-485e-9b16-82504ad2a4d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:22.804599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-19d9a7ba-094f-485e-9b16-82504ad2a4d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:22.814084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9a7ba-094f-485e-9b16-82504ad2a4d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:22.815549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9a7ba-094f-485e-9b16-82504ad2a4d8 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:22.824465 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1369/2746] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:48:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:48:32.820018 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=46 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:48:34.005935 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:34.008307 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:34.109298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31879030-7247-4c2e-9563-1fd422db7ed6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:34.116963 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40935e8-8dc6-4ea0-a9bb-f4cc96614f5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdcd6d09fee45c696449641ea36eae3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:34.118211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40935e8-8dc6-4ea0-a9bb-f4cc96614f5a service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:34.118211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40935e8-8dc6-4ea0-a9bb-f4cc96614f5a service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:34.118211 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40935e8-8dc6-4ea0-a9bb-f4cc96614f5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:34.120116 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e40935e8-8dc6-4ea0-a9bb-f4cc96614f5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:34.135465 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40935e8-8dc6-4ea0-a9bb-f4cc96614f5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:34.137408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40935e8-8dc6-4ea0-a9bb-f4cc96614f5a service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:34.149862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1378/2747] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:48:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:48:42.122387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9a7ba-094f-485e-9b16-82504ad2a4d8 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:42.126552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-19d9a7ba-094f-485e-9b16-82504ad2a4d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:42.183019 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40935e8-8dc6-4ea0-a9bb-f4cc96614f5a service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:42.185285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e40935e8-8dc6-4ea0-a9bb-f4cc96614f5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:42.214516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9a7ba-094f-485e-9b16-82504ad2a4d8 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:42.219577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5800a-b512-4883-8bca-72f601a39039 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928c5d1284b74f1291c031323bb03a20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:42.220195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5800a-b512-4883-8bca-72f601a39039 service placement] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:42.220481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5800a-b512-4883-8bca-72f601a39039 service placement] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:42.220754 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5800a-b512-4883-8bca-72f601a39039 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:42.222493 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-10b5800a-b512-4883-8bca-72f601a39039 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:42.235327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5800a-b512-4883-8bca-72f601a39039 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:42.236909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5800a-b512-4883-8bca-72f601a39039 service placement] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:42.248111 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1370/2748] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:48:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:48:42.262763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40935e8-8dc6-4ea0-a9bb-f4cc96614f5a service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:42.268762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea2b88-a405-40a8-aa59-9541ad74d78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:42.269503 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea2b88-a405-40a8-aa59-9541ad74d78e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:42.269857 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea2b88-a405-40a8-aa59-9541ad74d78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:42.270204 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea2b88-a405-40a8-aa59-9541ad74d78e admin admin] PATH_INFO: `/v3/users/1ba46a929ed54c51a33662367289eee5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:42.271501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea2b88-a405-40a8-aa59-9541ad74d78e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ba46a929ed54c51a33662367289eee5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:42.273483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea2b88-a405-40a8-aa59-9541ad74d78e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:42.321546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-16ea2b88-a405-40a8-aa59-9541ad74d78e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ba46a929ed54c51a33662367289eee5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:42.328922 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-16ea2b88-a405-40a8-aa59-9541ad74d78e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ba46a929ed54c51a33662367289eee5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:42.340882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-16ea2b88-a405-40a8-aa59-9541ad74d78e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ba46a929ed54c51a33662367289eee5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:42.341455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-16ea2b88-a405-40a8-aa59-9541ad74d78e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ba46a929ed54c51a33662367289eee5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:42.342055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-16ea2b88-a405-40a8-aa59-9541ad74d78e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ba46a929ed54c51a33662367289eee5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:42.354132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1379/2749] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:48:42 2022] DELETE /identity/v3/users/1ba46a929ed54c51a33662367289eee5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:48:42.528871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5800a-b512-4883-8bca-72f601a39039 service placement] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:42.539668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5800a-b512-4883-8bca-72f601a39039 service placement] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:42.597993 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb80c88-4be9-43e2-a54b-163318c769e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:42.598640 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb80c88-4be9-43e2-a54b-163318c769e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:42.599459 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb80c88-4be9-43e2-a54b-163318c769e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:42.599779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb80c88-4be9-43e2-a54b-163318c769e7 admin admin] PATH_INFO: `/v3/projects/49e0013064fc495285dad557b0beef7d` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:42.600922 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb80c88-4be9-43e2-a54b-163318c769e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49e0013064fc495285dad557b0beef7d)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:42.602825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb80c88-4be9-43e2-a54b-163318c769e7 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:42.650209 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0bb80c88-4be9-43e2-a54b-163318c769e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49e0013064fc495285dad557b0beef7d', 'request_id': 'req-0bb80c88-4be9-43e2-a54b-163318c769e7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:42.655716 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0bb80c88-4be9-43e2-a54b-163318c769e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49e0013064fc495285dad557b0beef7d', 'request_id': 'req-0bb80c88-4be9-43e2-a54b-163318c769e7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:42.661698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0bb80c88-4be9-43e2-a54b-163318c769e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49e0013064fc495285dad557b0beef7d', 'request_id': 'req-0bb80c88-4be9-43e2-a54b-163318c769e7'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:42.673931 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0bb80c88-4be9-43e2-a54b-163318c769e7 admin admin] The token cache is being invalidate because project 49e0013064fc495285dad557b0beef7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:48:42.674227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0bb80c88-4be9-43e2-a54b-163318c769e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:42.674898 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-0bb80c88-4be9-43e2-a54b-163318c769e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:42.676434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1371/2750] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:48:42 2022] DELETE /identity/v3/projects/49e0013064fc495285dad557b0beef7d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:48:42.681363 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47ddd6-62d2-4283-b057-a395393c04a9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:42.681655 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47ddd6-62d2-4283-b057-a395393c04a9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:42.681940 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47ddd6-62d2-4283-b057-a395393c04a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:42.691032 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:48:42.703598 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-aa47ddd6-62d2-4283-b057-a395393c04a9 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:48:42.704847 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-aa47ddd6-62d2-4283-b057-a395393c04a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:42.762322 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1380/2751] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:48:42.770167 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fdb8b-52fd-4c29-83ae-0fad97e0871a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:42.770465 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fdb8b-52fd-4c29-83ae-0fad97e0871a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:42.770754 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fdb8b-52fd-4c29-83ae-0fad97e0871a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:42.791675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-442fdb8b-52fd-4c29-83ae-0fad97e0871a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:48:42.792928 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-442fdb8b-52fd-4c29-83ae-0fad97e0871a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:42.800659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1372/2752] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:48:42.804853 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47ddd6-62d2-4283-b057-a395393c04a9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:42.815035 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47ddd6-62d2-4283-b057-a395393c04a9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:42.819618 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535c61d-29e4-4797-b8f1-03e47710c2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:42.820226 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535c61d-29e4-4797-b8f1-03e47710c2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:42.820511 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535c61d-29e4-4797-b8f1-03e47710c2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:42.820792 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535c61d-29e4-4797-b8f1-03e47710c2d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:42.821487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535c61d-29e4-4797-b8f1-03e47710c2d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:48:42.821801 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535c61d-29e4-4797-b8f1-03e47710c2d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:42.823435 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535c61d-29e4-4797-b8f1-03e47710c2d0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:42.835375 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1381/2753] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:48:42.839537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fdb8b-52fd-4c29-83ae-0fad97e0871a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:42.848659 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fdb8b-52fd-4c29-83ae-0fad97e0871a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:42.853169 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7147874-9f28-45fb-9b3f-5831e435b94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:42.853761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7147874-9f28-45fb-9b3f-5831e435b94b admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:42.854043 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7147874-9f28-45fb-9b3f-5831e435b94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:42.854315 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7147874-9f28-45fb-9b3f-5831e435b94b admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:42.854963 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7147874-9f28-45fb-9b3f-5831e435b94b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:42.856919 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7147874-9f28-45fb-9b3f-5831e435b94b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:42.867670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:48:42.894840 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1373/2754] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:48:42 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:48:42.899284 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535c61d-29e4-4797-b8f1-03e47710c2d0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:42.909292 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535c61d-29e4-4797-b8f1-03e47710c2d0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:42.968683 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f6ae6-925c-43f1-95e8-0fc99ad583c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:42.969294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f6ae6-925c-43f1-95e8-0fc99ad583c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:42.969579 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f6ae6-925c-43f1-95e8-0fc99ad583c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:42.969856 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f6ae6-925c-43f1-95e8-0fc99ad583c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:42.970523 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f6ae6-925c-43f1-95e8-0fc99ad583c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:42.972620 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f6ae6-925c-43f1-95e8-0fc99ad583c6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:42.996377 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1382/2755] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:48:42 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:48:42.998451 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7147874-9f28-45fb-9b3f-5831e435b94b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:43.008997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7147874-9f28-45fb-9b3f-5831e435b94b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:43.013917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b614ac8-0dfe-4938-b342-3753fbae6a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:43.014547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b614ac8-0dfe-4938-b342-3753fbae6a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:43.014827 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b614ac8-0dfe-4938-b342-3753fbae6a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:43.015123 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b614ac8-0dfe-4938-b342-3753fbae6a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:43.015806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b614ac8-0dfe-4938-b342-3753fbae6a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:43.017386 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b614ac8-0dfe-4938-b342-3753fbae6a50 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:43.027282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1374/2756] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:48:43.032492 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f6ae6-925c-43f1-95e8-0fc99ad583c6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:43.042379 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f6ae6-925c-43f1-95e8-0fc99ad583c6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:43.047623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2685d47-99df-4f7a-a9e0-0cbf11eb9a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:43.048225 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2685d47-99df-4f7a-a9e0-0cbf11eb9a69 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:43.048514 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2685d47-99df-4f7a-a9e0-0cbf11eb9a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:43.048796 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2685d47-99df-4f7a-a9e0-0cbf11eb9a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:43.049485 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2685d47-99df-4f7a-a9e0-0cbf11eb9a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:43.051095 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2685d47-99df-4f7a-a9e0-0cbf11eb9a69 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:43.062429 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1383/2757] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:48:43.066695 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b614ac8-0dfe-4938-b342-3753fbae6a50 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:43.077088 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b614ac8-0dfe-4938-b342-3753fbae6a50 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:43.081784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39e72a-ca6d-4445-ae76-d37d125f4b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:43.082433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39e72a-ca6d-4445-ae76-d37d125f4b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:43.082718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39e72a-ca6d-4445-ae76-d37d125f4b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:43.083001 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39e72a-ca6d-4445-ae76-d37d125f4b19 admin admin] PATH_INFO: `/v3/projects/2ab1b8842fa844109b04c83970085746/users/d80777131f864995911abf29b1b65fb6/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:43.097551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39e72a-ca6d-4445-ae76-d37d125f4b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab1b8842fa844109b04c83970085746, user_id=d80777131f864995911abf29b1b65fb6, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:43.100248 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39e72a-ca6d-4445-ae76-d37d125f4b19 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:43.113818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1375/2758] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:48:43 2022] PUT /identity/v3/projects/2ab1b8842fa844109b04c83970085746/users/d80777131f864995911abf29b1b65fb6/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:48:43.124600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd1589-4c4d-450f-8a60-2994b38b4492 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:43.125144 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd1589-4c4d-450f-8a60-2994b38b4492 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:43.125426 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd1589-4c4d-450f-8a60-2994b38b4492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:43.152794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3bdd1589-4c4d-450f-8a60-2994b38b4492 None None] MFA Rules not processed for user `d80777131f864995911abf29b1b65fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:48:43.153855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3bdd1589-4c4d-450f-8a60-2994b38b4492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:43.215932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:48:43.220325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1384/2759] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:48:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:48:43.231777 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39e72a-ca6d-4445-ae76-d37d125f4b19 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:43.233635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ce39e72a-ca6d-4445-ae76-d37d125f4b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:43.298268 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39e72a-ca6d-4445-ae76-d37d125f4b19 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:43.302023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c45f1-6898-47d7-84b6-fb33f36dabc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:43.302642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c45f1-6898-47d7-84b6-fb33f36dabc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:43.302930 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c45f1-6898-47d7-84b6-fb33f36dabc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:43.303249 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c45f1-6898-47d7-84b6-fb33f36dabc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:43.312290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c45f1-6898-47d7-84b6-fb33f36dabc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:43.313719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c45f1-6898-47d7-84b6-fb33f36dabc5 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:43.322631 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1376/2760] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:48:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:48:51.660342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd1589-4c4d-450f-8a60-2994b38b4492 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:51.723223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd1589-4c4d-450f-8a60-2994b38b4492 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:51.727666 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:51.728301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:51.728581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:51.728856 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] PATH_INFO: `/v3/users/d80777131f864995911abf29b1b65fb6` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:51.730082 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d80777131f864995911abf29b1b65fb6)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:51.732202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:51.782512 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd80777131f864995911abf29b1b65fb6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:51.790337 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd80777131f864995911abf29b1b65fb6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:51.802237 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd80777131f864995911abf29b1b65fb6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:51.802875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd80777131f864995911abf29b1b65fb6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:51.803515 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd80777131f864995911abf29b1b65fb6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:51.816067 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1385/2761] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:48:51 2022] DELETE /identity/v3/users/d80777131f864995911abf29b1b65fb6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:48:51.820636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bf12b-ce84-4e1f-b311-66f764be3418 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:51.821112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bf12b-ce84-4e1f-b311-66f764be3418 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:51.821226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bf12b-ce84-4e1f-b311-66f764be3418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:51.841124 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-5a2bf12b-ce84-4e1f-b311-66f764be3418 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:48:51.842330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5a2bf12b-ce84-4e1f-b311-66f764be3418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:51.901010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1377/2762] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:48:51.911827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:51.913304 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:51.970941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee04ac-d4f9-4e16-b018-345ce21fe3bc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:51.974612 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f056959-fa8e-4a8a-9e55-49661e6f9aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:51.975299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f056959-fa8e-4a8a-9e55-49661e6f9aca service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:51.975585 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f056959-fa8e-4a8a-9e55-49661e6f9aca service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:51.975864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f056959-fa8e-4a8a-9e55-49661e6f9aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:51.985411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f056959-fa8e-4a8a-9e55-49661e6f9aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:51.986874 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f056959-fa8e-4a8a-9e55-49661e6f9aca service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:51.997341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1386/2763] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:48:52.256083 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2bf12b-ce84-4e1f-b311-66f764be3418 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:52.257562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5a2bf12b-ce84-4e1f-b311-66f764be3418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:52.269692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2bf12b-ce84-4e1f-b311-66f764be3418 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:52.274740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:52.275874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:52.275874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:52.276066 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6 admin admin] PATH_INFO: `/v3/projects/2ab1b8842fa844109b04c83970085746` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:52.277759 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab1b8842fa844109b04c83970085746)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:52.279781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:52.324621 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:48:52.327864 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab1b8842fa844109b04c83970085746', 'request_id': 'req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:52.334714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab1b8842fa844109b04c83970085746', 'request_id': 'req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:52.341723 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab1b8842fa844109b04c83970085746', 'request_id': 'req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:52.353469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6 admin admin] The token cache is being invalidate because project 2ab1b8842fa844109b04c83970085746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:48:52.353757 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:52.354375 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-ba0f5fd8-0755-4c2a-8628-01ab453b3ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:48:52.356740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1378/2764] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:48:52 2022] DELETE /identity/v3/projects/2ab1b8842fa844109b04c83970085746 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:48:52.361632 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dafa5-0f55-4eb5-8aab-a5cc48543058 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:52.361921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dafa5-0f55-4eb5-8aab-a5cc48543058 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:52.362202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dafa5-0f55-4eb5-8aab-a5cc48543058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:52.381486 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ab9dafa5-0f55-4eb5-8aab-a5cc48543058 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:48:52.382686 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ab9dafa5-0f55-4eb5-8aab-a5cc48543058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:52.399744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:48:52.446340 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1387/2765] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:48:52.454118 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe9e81-1303-41e8-a267-bd2af1dfa458 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:52.454404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe9e81-1303-41e8-a267-bd2af1dfa458 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:52.454680 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe9e81-1303-41e8-a267-bd2af1dfa458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:52.474283 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-01fe9e81-1303-41e8-a267-bd2af1dfa458 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:48:52.475518 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-01fe9e81-1303-41e8-a267-bd2af1dfa458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:52.484257 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1379/2766] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:48:52.488604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9dafa5-0f55-4eb5-8aab-a5cc48543058 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:52.498365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9dafa5-0f55-4eb5-8aab-a5cc48543058 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:52.503260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1501e70-c0df-4bfa-bace-36675d3ba9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:52.503864 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1501e70-c0df-4bfa-bace-36675d3ba9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:52.504154 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1501e70-c0df-4bfa-bace-36675d3ba9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:52.504435 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1501e70-c0df-4bfa-bace-36675d3ba9e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:52.505127 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1501e70-c0df-4bfa-bace-36675d3ba9e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:48:52.505439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1501e70-c0df-4bfa-bace-36675d3ba9e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:52.507042 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1501e70-c0df-4bfa-bace-36675d3ba9e7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:52.518566 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1388/2767] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:48:52.522596 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe9e81-1303-41e8-a267-bd2af1dfa458 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:52.532242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe9e81-1303-41e8-a267-bd2af1dfa458 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:52.536837 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88c45e-e136-415c-8867-886e6f1a2ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:52.537432 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88c45e-e136-415c-8867-886e6f1a2ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:52.537713 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88c45e-e136-415c-8867-886e6f1a2ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:52.537986 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88c45e-e136-415c-8867-886e6f1a2ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:52.538644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88c45e-e136-415c-8867-886e6f1a2ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:52.540563 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88c45e-e136-415c-8867-886e6f1a2ec1 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:52.568722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1380/2768] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:48:52 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:48:52.573179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1501e70-c0df-4bfa-bace-36675d3ba9e7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:52.584051 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1501e70-c0df-4bfa-bace-36675d3ba9e7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:52.639978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eade25-59bd-4469-83ad-f2abb385be4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:52.640587 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eade25-59bd-4469-83ad-f2abb385be4b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:52.640875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eade25-59bd-4469-83ad-f2abb385be4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:52.641157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eade25-59bd-4469-83ad-f2abb385be4b admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:52.641826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eade25-59bd-4469-83ad-f2abb385be4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:52.643781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eade25-59bd-4469-83ad-f2abb385be4b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:52.664692 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1389/2769] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:48:52 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:48:52.669815 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88c45e-e136-415c-8867-886e6f1a2ec1 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:52.680293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88c45e-e136-415c-8867-886e6f1a2ec1 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:52.686153 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ac191-d151-4ad7-9d5d-0eaf614339e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:52.686762 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ac191-d151-4ad7-9d5d-0eaf614339e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:52.687051 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ac191-d151-4ad7-9d5d-0eaf614339e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:52.687354 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ac191-d151-4ad7-9d5d-0eaf614339e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:52.688068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ac191-d151-4ad7-9d5d-0eaf614339e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:52.689649 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ac191-d151-4ad7-9d5d-0eaf614339e6 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:52.700446 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1381/2770] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:48:52.704428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eade25-59bd-4469-83ad-f2abb385be4b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:52.714415 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eade25-59bd-4469-83ad-f2abb385be4b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:52.719131 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff021a-e311-4b67-8604-e287626b4ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:52.719738 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff021a-e311-4b67-8604-e287626b4ffc admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:52.720027 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff021a-e311-4b67-8604-e287626b4ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:52.720309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff021a-e311-4b67-8604-e287626b4ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:52.721000 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff021a-e311-4b67-8604-e287626b4ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:52.722586 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff021a-e311-4b67-8604-e287626b4ffc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:52.733849 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1390/2771] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:48:52.737801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ac191-d151-4ad7-9d5d-0eaf614339e6 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:52.748324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ac191-d151-4ad7-9d5d-0eaf614339e6 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:52.753110 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7e2cd-6a8f-4ecd-b7be-34755555376f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:52.753758 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7e2cd-6a8f-4ecd-b7be-34755555376f admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:52.754043 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7e2cd-6a8f-4ecd-b7be-34755555376f admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:52.754324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7e2cd-6a8f-4ecd-b7be-34755555376f admin admin] PATH_INFO: `/v3/projects/628e02ab347e44a7bd14e70b8a538150/users/09c70204a8294c86a2475a3576163722/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:52.768907 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7e2cd-6a8f-4ecd-b7be-34755555376f admin admin] RBAC: Authorizing `identity:create_grant(project_id=628e02ab347e44a7bd14e70b8a538150, user_id=09c70204a8294c86a2475a3576163722, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:52.771590 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7e2cd-6a8f-4ecd-b7be-34755555376f admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:52.784722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1382/2772] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:48:52 2022] PUT /identity/v3/projects/628e02ab347e44a7bd14e70b8a538150/users/09c70204a8294c86a2475a3576163722/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:48:52.796980 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a230e-3283-419c-a84a-3b88dc1c4540 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:52.797273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a230e-3283-419c-a84a-3b88dc1c4540 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:52.797560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a230e-3283-419c-a84a-3b88dc1c4540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:52.825394 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-714a230e-3283-419c-a84a-3b88dc1c4540 None None] MFA Rules not processed for user `09c70204a8294c86a2475a3576163722`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:48:52.826598 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-714a230e-3283-419c-a84a-3b88dc1c4540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:52.888232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1391/2773] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:48:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:48:52.892785 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7e2cd-6a8f-4ecd-b7be-34755555376f admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:52.955139 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7e2cd-6a8f-4ecd-b7be-34755555376f admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:52.959683 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93756485-c3fc-428d-a61f-9e1326ee2620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:52.960294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93756485-c3fc-428d-a61f-9e1326ee2620 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:52.960576 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93756485-c3fc-428d-a61f-9e1326ee2620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:52.960853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93756485-c3fc-428d-a61f-9e1326ee2620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:52.961520 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93756485-c3fc-428d-a61f-9e1326ee2620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:52.963492 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93756485-c3fc-428d-a61f-9e1326ee2620 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:52.990324 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1383/2774] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:48:52 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:48:52.994450 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a230e-3283-419c-a84a-3b88dc1c4540 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:53.005820 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a230e-3283-419c-a84a-3b88dc1c4540 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:53.017850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:48:53.061223 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485a2a9-4cae-4219-8266-842122d3c2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:53.061829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485a2a9-4cae-4219-8266-842122d3c2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:53.062109 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485a2a9-4cae-4219-8266-842122d3c2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:53.062390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485a2a9-4cae-4219-8266-842122d3c2f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:53.063046 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485a2a9-4cae-4219-8266-842122d3c2f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:53.064978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485a2a9-4cae-4219-8266-842122d3c2f2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:53.085280 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1392/2775] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:48:52 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:48:53.089272 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93756485-c3fc-428d-a61f-9e1326ee2620 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:53.099894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93756485-c3fc-428d-a61f-9e1326ee2620 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:53.104433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e00278-c14b-42e7-a384-8f5f82017446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:53.105032 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e00278-c14b-42e7-a384-8f5f82017446 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:53.105963 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e00278-c14b-42e7-a384-8f5f82017446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:53.106262 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e00278-c14b-42e7-a384-8f5f82017446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:53.107013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e00278-c14b-42e7-a384-8f5f82017446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:53.108665 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e00278-c14b-42e7-a384-8f5f82017446 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:53.118951 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1384/2776] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:48:53.122937 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485a2a9-4cae-4219-8266-842122d3c2f2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:53.133047 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485a2a9-4cae-4219-8266-842122d3c2f2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:53.138092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5677029-2930-4d19-b5bc-85b0e7920516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:53.138738 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5677029-2930-4d19-b5bc-85b0e7920516 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:53.139022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5677029-2930-4d19-b5bc-85b0e7920516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:53.139334 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5677029-2930-4d19-b5bc-85b0e7920516 admin admin] PATH_INFO: `/v3/projects/5aac3ed7e8f144889a32473cd813c84e/users/a6cfd7e830db42b9bb0b5cd257fa5548/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:53.154391 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5677029-2930-4d19-b5bc-85b0e7920516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aac3ed7e8f144889a32473cd813c84e, user_id=a6cfd7e830db42b9bb0b5cd257fa5548, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:53.157134 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5677029-2930-4d19-b5bc-85b0e7920516 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:53.170818 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1393/2777] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:48:53 2022] PUT /identity/v3/projects/5aac3ed7e8f144889a32473cd813c84e/users/a6cfd7e830db42b9bb0b5cd257fa5548/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:48:53.174812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e00278-c14b-42e7-a384-8f5f82017446 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:53.184806 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e00278-c14b-42e7-a384-8f5f82017446 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:53.227894 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:48:53.245028 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f5f80-6f6a-41af-99ed-bae12a723d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:53.245651 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f5f80-6f6a-41af-99ed-bae12a723d5b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:53.246040 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f5f80-6f6a-41af-99ed-bae12a723d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:53.246326 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f5f80-6f6a-41af-99ed-bae12a723d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:53.247019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f5f80-6f6a-41af-99ed-bae12a723d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:53.248653 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f5f80-6f6a-41af-99ed-bae12a723d5b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:53.258646 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1385/2778] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:48:53.263253 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5677029-2930-4d19-b5bc-85b0e7920516 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:53.273464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5677029-2930-4d19-b5bc-85b0e7920516 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:53.278551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4694952-e156-4c47-b215-ecf63524fa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:53.279236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4694952-e156-4c47-b215-ecf63524fa92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:53.279519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4694952-e156-4c47-b215-ecf63524fa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:53.279794 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4694952-e156-4c47-b215-ecf63524fa92 admin admin] PATH_INFO: `/v3/projects/5aac3ed7e8f144889a32473cd813c84e/users/a6cfd7e830db42b9bb0b5cd257fa5548/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:53.281770 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4694952-e156-4c47-b215-ecf63524fa92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aac3ed7e8f144889a32473cd813c84e, user_id=a6cfd7e830db42b9bb0b5cd257fa5548, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:53.284430 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4694952-e156-4c47-b215-ecf63524fa92 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:53.295002 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1394/2779] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:48:53 2022] PUT /identity/v3/projects/5aac3ed7e8f144889a32473cd813c84e/users/a6cfd7e830db42b9bb0b5cd257fa5548/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:48:53.305020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493051a-1ac6-4883-a795-d453ed3671e1 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:53.305311 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493051a-1ac6-4883-a795-d453ed3671e1 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:53.305592 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493051a-1ac6-4883-a795-d453ed3671e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:53.337290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-7493051a-1ac6-4883-a795-d453ed3671e1 None None] MFA Rules not processed for user `a6cfd7e830db42b9bb0b5cd257fa5548`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:48:53.338516 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-7493051a-1ac6-4883-a795-d453ed3671e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:53.406970 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1386/2780] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:48:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:48:53.418348 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4694952-e156-4c47-b215-ecf63524fa92 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:53.420162 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a4694952-e156-4c47-b215-ecf63524fa92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:48:53.477124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4694952-e156-4c47-b215-ecf63524fa92 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:53.480641 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8de47-f434-4a12-8703-9b337ebcac7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:53.481228 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8de47-f434-4a12-8703-9b337ebcac7b service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:53.481508 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8de47-f434-4a12-8703-9b337ebcac7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:53.481782 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8de47-f434-4a12-8703-9b337ebcac7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:53.490534 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8de47-f434-4a12-8703-9b337ebcac7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:53.491973 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8de47-f434-4a12-8703-9b337ebcac7b service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:53.501400 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1395/2781] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:48:53 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:48:53.573615 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493051a-1ac6-4883-a795-d453ed3671e1 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:48:53.583856 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493051a-1ac6-4883-a795-d453ed3671e1 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:48:53.587413 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd2076-e856-4f8f-b6d1-bb4850027566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:48:53.588014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd2076-e856-4f8f-b6d1-bb4850027566 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:48:53.588294 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd2076-e856-4f8f-b6d1-bb4850027566 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:48:53.588567 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd2076-e856-4f8f-b6d1-bb4850027566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:48:53.645693 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd2076-e856-4f8f-b6d1-bb4850027566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:48:53.647144 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd2076-e856-4f8f-b6d1-bb4850027566 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:48:53.656764 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1387/2782] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:48:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:03.497054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:03.651953 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=71 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:34.201124 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8de47-f434-4a12-8703-9b337ebcac7b service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:34.261380 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8de47-f434-4a12-8703-9b337ebcac7b service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:34.265861 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:34.266485 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:34.267418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:34.267697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] PATH_INFO: `/v3/users/09c70204a8294c86a2475a3576163722` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:34.268857 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c70204a8294c86a2475a3576163722)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:34.270760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:34.314805 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c70204a8294c86a2475a3576163722'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:34.321833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c70204a8294c86a2475a3576163722'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:34.338852 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c70204a8294c86a2475a3576163722'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:34.339552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c70204a8294c86a2475a3576163722'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:34.340155 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c70204a8294c86a2475a3576163722'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:34.352241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1396/2783] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:49:34 2022] DELETE /identity/v3/users/09c70204a8294c86a2475a3576163722 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:34.356812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762ce40-fe98-44d3-98c5-99ede28f7ab8 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:34.357297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762ce40-fe98-44d3-98c5-99ede28f7ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:34.357406 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762ce40-fe98-44d3-98c5-99ede28f7ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:34.376560 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-e762ce40-fe98-44d3-98c5-99ede28f7ab8 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:34.377923 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e762ce40-fe98-44d3-98c5-99ede28f7ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:34.434765 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1388/2784] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:34.445132 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:34.446532 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:34.502309 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6e1b9-2840-4c7c-a95e-4504bfa00dc8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:34.505905 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a38dee-f274-4c9c-ab77-bfe9b166fe2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:34.506519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a38dee-f274-4c9c-ab77-bfe9b166fe2c service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:34.506788 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a38dee-f274-4c9c-ab77-bfe9b166fe2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:34.507084 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a38dee-f274-4c9c-ab77-bfe9b166fe2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:34.516231 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a38dee-f274-4c9c-ab77-bfe9b166fe2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:34.518572 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a38dee-f274-4c9c-ab77-bfe9b166fe2c service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:34.527947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1397/2785] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:34.796981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762ce40-fe98-44d3-98c5-99ede28f7ab8 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:34.799015 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-e762ce40-fe98-44d3-98c5-99ede28f7ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:34.811334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762ce40-fe98-44d3-98c5-99ede28f7ab8 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:34.818282 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:34.818932 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:34.819256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:34.819540 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] PATH_INFO: `/v3/projects/628e02ab347e44a7bd14e70b8a538150` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:34.820714 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=628e02ab347e44a7bd14e70b8a538150)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:34.822732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:34.870136 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '628e02ab347e44a7bd14e70b8a538150', 'request_id': 'req-63d78dd8-f0f1-42ab-a8c5-7f4698328438'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:34.875789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '628e02ab347e44a7bd14e70b8a538150', 'request_id': 'req-63d78dd8-f0f1-42ab-a8c5-7f4698328438'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:34.882974 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '628e02ab347e44a7bd14e70b8a538150', 'request_id': 'req-63d78dd8-f0f1-42ab-a8c5-7f4698328438'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:34.896310 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] The token cache is being invalidate because project 628e02ab347e44a7bd14e70b8a538150 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:49:34.896601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:34.897221 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:34.898719 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1389/2786] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:49:34 2022] DELETE /identity/v3/projects/628e02ab347e44a7bd14e70b8a538150 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:34.902761 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a38dee-f274-4c9c-ab77-bfe9b166fe2c service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:34.913470 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a38dee-f274-4c9c-ab77-bfe9b166fe2c service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:34.932703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:34.975227 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249755e-c9e5-4a45-9aab-9ee7631c4ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:34.975880 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249755e-c9e5-4a45-9aab-9ee7631c4ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:34.976179 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249755e-c9e5-4a45-9aab-9ee7631c4ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:34.976442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249755e-c9e5-4a45-9aab-9ee7631c4ecb admin admin] PATH_INFO: `/v3/users/a6cfd7e830db42b9bb0b5cd257fa5548` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:34.977602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249755e-c9e5-4a45-9aab-9ee7631c4ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6cfd7e830db42b9bb0b5cd257fa5548)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:34.979550 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249755e-c9e5-4a45-9aab-9ee7631c4ecb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:35.026690 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1249755e-c9e5-4a45-9aab-9ee7631c4ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6cfd7e830db42b9bb0b5cd257fa5548'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:35.034380 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1249755e-c9e5-4a45-9aab-9ee7631c4ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6cfd7e830db42b9bb0b5cd257fa5548'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:35.046551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1249755e-c9e5-4a45-9aab-9ee7631c4ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6cfd7e830db42b9bb0b5cd257fa5548'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:35.047229 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1249755e-c9e5-4a45-9aab-9ee7631c4ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6cfd7e830db42b9bb0b5cd257fa5548'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:35.047891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1249755e-c9e5-4a45-9aab-9ee7631c4ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6cfd7e830db42b9bb0b5cd257fa5548'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:35.060452 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1398/2787] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:49:34 2022] DELETE /identity/v3/users/a6cfd7e830db42b9bb0b5cd257fa5548 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:35.327344 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:35.328784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:35.344095 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d78dd8-f0f1-42ab-a8c5-7f4698328438 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:35.403150 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99babf9-747e-4411-9873-965fe27515de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:35.403775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99babf9-747e-4411-9873-965fe27515de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:35.404013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99babf9-747e-4411-9873-965fe27515de admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:35.404295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99babf9-747e-4411-9873-965fe27515de admin admin] PATH_INFO: `/v3/projects/5aac3ed7e8f144889a32473cd813c84e` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:35.405442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99babf9-747e-4411-9873-965fe27515de admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aac3ed7e8f144889a32473cd813c84e)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:35.408066 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99babf9-747e-4411-9873-965fe27515de admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:35.437629 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:35.456213 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c99babf9-747e-4411-9873-965fe27515de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aac3ed7e8f144889a32473cd813c84e', 'request_id': 'req-c99babf9-747e-4411-9873-965fe27515de'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:35.461809 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c99babf9-747e-4411-9873-965fe27515de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aac3ed7e8f144889a32473cd813c84e', 'request_id': 'req-c99babf9-747e-4411-9873-965fe27515de'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:35.467859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c99babf9-747e-4411-9873-965fe27515de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aac3ed7e8f144889a32473cd813c84e', 'request_id': 'req-c99babf9-747e-4411-9873-965fe27515de'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:35.480552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c99babf9-747e-4411-9873-965fe27515de admin admin] The token cache is being invalidate because project 5aac3ed7e8f144889a32473cd813c84e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:49:35.480844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c99babf9-747e-4411-9873-965fe27515de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:35.481480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-c99babf9-747e-4411-9873-965fe27515de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:35.483029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1390/2788] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:49:35 2022] DELETE /identity/v3/projects/5aac3ed7e8f144889a32473cd813c84e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:35.487943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41441d-3be7-4b93-bb6a-8ae902e3fe93 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:35.488402 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41441d-3be7-4b93-bb6a-8ae902e3fe93 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:35.488521 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41441d-3be7-4b93-bb6a-8ae902e3fe93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:35.508150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ef41441d-3be7-4b93-bb6a-8ae902e3fe93 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:35.509385 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ef41441d-3be7-4b93-bb6a-8ae902e3fe93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:35.566813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1399/2789] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:35.574489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208369ee-dfe0-46f6-92ea-ef1e6d3b9240 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:35.574774 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208369ee-dfe0-46f6-92ea-ef1e6d3b9240 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:35.575058 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208369ee-dfe0-46f6-92ea-ef1e6d3b9240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:35.596224 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-208369ee-dfe0-46f6-92ea-ef1e6d3b9240 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:35.597470 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-208369ee-dfe0-46f6-92ea-ef1e6d3b9240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:35.605346 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1391/2790] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:35.609482 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41441d-3be7-4b93-bb6a-8ae902e3fe93 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:35.619164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41441d-3be7-4b93-bb6a-8ae902e3fe93 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:35.623675 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a52ee-265d-47e0-9bcf-dd9230eeb2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:35.624278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a52ee-265d-47e0-9bcf-dd9230eeb2ed admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:35.624560 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a52ee-265d-47e0-9bcf-dd9230eeb2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:35.624837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a52ee-265d-47e0-9bcf-dd9230eeb2ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:35.625524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a52ee-265d-47e0-9bcf-dd9230eeb2ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:49:35.625831 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a52ee-265d-47e0-9bcf-dd9230eeb2ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:35.627763 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a52ee-265d-47e0-9bcf-dd9230eeb2ed admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:35.639111 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1400/2791] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:49:35.643909 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208369ee-dfe0-46f6-92ea-ef1e6d3b9240 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:35.654262 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208369ee-dfe0-46f6-92ea-ef1e6d3b9240 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:35.659513 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffee418-5c47-4b20-8243-3fc0bbcb3554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:35.660113 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffee418-5c47-4b20-8243-3fc0bbcb3554 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:35.660394 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffee418-5c47-4b20-8243-3fc0bbcb3554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:35.660668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffee418-5c47-4b20-8243-3fc0bbcb3554 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:35.661342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffee418-5c47-4b20-8243-3fc0bbcb3554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:35.663261 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffee418-5c47-4b20-8243-3fc0bbcb3554 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:35.691226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1392/2792] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:49:35 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:35.695552 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a52ee-265d-47e0-9bcf-dd9230eeb2ed admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:35.706297 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a52ee-265d-47e0-9bcf-dd9230eeb2ed admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:35.732661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:35.765352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fb215-59b4-4a92-9072-85f2aef1b1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:35.765955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fb215-59b4-4a92-9072-85f2aef1b1cb admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:35.766239 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fb215-59b4-4a92-9072-85f2aef1b1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:35.766598 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fb215-59b4-4a92-9072-85f2aef1b1cb admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:35.768250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fb215-59b4-4a92-9072-85f2aef1b1cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:35.770173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fb215-59b4-4a92-9072-85f2aef1b1cb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:35.792960 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1401/2793] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:49:35 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:35.797098 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffee418-5c47-4b20-8243-3fc0bbcb3554 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:35.807330 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffee418-5c47-4b20-8243-3fc0bbcb3554 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:35.811845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ca751-88b0-4e04-95f3-09b108681298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:35.812440 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ca751-88b0-4e04-95f3-09b108681298 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:35.812738 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ca751-88b0-4e04-95f3-09b108681298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:35.812991 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ca751-88b0-4e04-95f3-09b108681298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:35.814819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ca751-88b0-4e04-95f3-09b108681298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:35.816507 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ca751-88b0-4e04-95f3-09b108681298 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:35.826434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1393/2794] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:35.831604 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fb215-59b4-4a92-9072-85f2aef1b1cb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:35.841624 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fb215-59b4-4a92-9072-85f2aef1b1cb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:35.846416 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfe5cd-b7c3-4b40-a9e6-d26ced701bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:35.847020 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfe5cd-b7c3-4b40-a9e6-d26ced701bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:35.847343 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfe5cd-b7c3-4b40-a9e6-d26ced701bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:35.847626 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfe5cd-b7c3-4b40-a9e6-d26ced701bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:35.848318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfe5cd-b7c3-4b40-a9e6-d26ced701bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:35.849916 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfe5cd-b7c3-4b40-a9e6-d26ced701bd4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:35.860403 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1402/2795] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:49:35.864528 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ca751-88b0-4e04-95f3-09b108681298 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:35.874636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ca751-88b0-4e04-95f3-09b108681298 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:35.880161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ab42c-36de-4a24-83e4-b17132589035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:35.880821 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ab42c-36de-4a24-83e4-b17132589035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:35.881109 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ab42c-36de-4a24-83e4-b17132589035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:35.881393 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ab42c-36de-4a24-83e4-b17132589035 admin admin] PATH_INFO: `/v3/projects/4af88dc9eb944ae3bd3b3ca8b3ca8546/users/dc05c3c30e724d69b419100bdbc28c97/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:35.896410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ab42c-36de-4a24-83e4-b17132589035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af88dc9eb944ae3bd3b3ca8b3ca8546, user_id=dc05c3c30e724d69b419100bdbc28c97, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:35.899107 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ab42c-36de-4a24-83e4-b17132589035 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:35.912072 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1394/2796] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:49:35 2022] PUT /identity/v3/projects/4af88dc9eb944ae3bd3b3ca8b3ca8546/users/dc05c3c30e724d69b419100bdbc28c97/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:35.922079 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613a8c2-a232-457c-88c2-7ba71aa30350 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:35.922372 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613a8c2-a232-457c-88c2-7ba71aa30350 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:35.922656 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613a8c2-a232-457c-88c2-7ba71aa30350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:35.951273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f613a8c2-a232-457c-88c2-7ba71aa30350 None None] MFA Rules not processed for user `dc05c3c30e724d69b419100bdbc28c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:35.952557 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f613a8c2-a232-457c-88c2-7ba71aa30350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:36.017365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1403/2797] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:36.021630 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ab42c-36de-4a24-83e4-b17132589035 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:36.084112 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ab42c-36de-4a24-83e4-b17132589035 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:36.088871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d1d6-bd7f-4ff1-853c-cff4497631b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:36.089482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d1d6-bd7f-4ff1-853c-cff4497631b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:36.089761 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d1d6-bd7f-4ff1-853c-cff4497631b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:36.090038 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d1d6-bd7f-4ff1-853c-cff4497631b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:36.090708 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2d1d6-bd7f-4ff1-853c-cff4497631b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:36.092650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2d1d6-bd7f-4ff1-853c-cff4497631b0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:36.118778 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1395/2798] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:49:36 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:36.122829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613a8c2-a232-457c-88c2-7ba71aa30350 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:36.134282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613a8c2-a232-457c-88c2-7ba71aa30350 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:36.190278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db2c98-99fc-43ac-860b-ac44d816ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:36.190913 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db2c98-99fc-43ac-860b-ac44d816ca23 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:36.191314 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db2c98-99fc-43ac-860b-ac44d816ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:36.191613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db2c98-99fc-43ac-860b-ac44d816ca23 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:36.192406 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db2c98-99fc-43ac-860b-ac44d816ca23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:36.194357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db2c98-99fc-43ac-860b-ac44d816ca23 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:36.215298 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1404/2799] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:49:36 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:49:36.219591 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d1d6-bd7f-4ff1-853c-cff4497631b0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:36.229781 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d1d6-bd7f-4ff1-853c-cff4497631b0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:36.234233 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec530c-57b3-474c-946d-02ff6534e2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:36.234828 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec530c-57b3-474c-946d-02ff6534e2eb admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:36.235134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec530c-57b3-474c-946d-02ff6534e2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:36.236012 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec530c-57b3-474c-946d-02ff6534e2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:36.236790 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec530c-57b3-474c-946d-02ff6534e2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:36.238370 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec530c-57b3-474c-946d-02ff6534e2eb admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:36.248721 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1396/2800] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:36.253812 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db2c98-99fc-43ac-860b-ac44d816ca23 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:36.263697 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db2c98-99fc-43ac-860b-ac44d816ca23 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:36.268551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7e0fb-b94e-461e-9d8f-f13dc9f47dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:36.269209 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7e0fb-b94e-461e-9d8f-f13dc9f47dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:36.269498 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7e0fb-b94e-461e-9d8f-f13dc9f47dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:36.269784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7e0fb-b94e-461e-9d8f-f13dc9f47dfe admin admin] PATH_INFO: `/v3/projects/dbf51e9bb72044e5a1b4dcbf0c597ce5/users/abc707a5af9a45ae92c1d8b9e7503d0b/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:36.284120 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7e0fb-b94e-461e-9d8f-f13dc9f47dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf51e9bb72044e5a1b4dcbf0c597ce5, user_id=abc707a5af9a45ae92c1d8b9e7503d0b, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:36.286771 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7e0fb-b94e-461e-9d8f-f13dc9f47dfe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:36.299670 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1405/2801] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:49:36 2022] PUT /identity/v3/projects/dbf51e9bb72044e5a1b4dcbf0c597ce5/users/abc707a5af9a45ae92c1d8b9e7503d0b/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:36.303564 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec530c-57b3-474c-946d-02ff6534e2eb admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:36.314500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec530c-57b3-474c-946d-02ff6534e2eb admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:36.333939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:36.371114 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda2b9c-2fab-4fe0-9cb8-d8467d710f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:36.371732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda2b9c-2fab-4fe0-9cb8-d8467d710f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:36.372014 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda2b9c-2fab-4fe0-9cb8-d8467d710f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:36.372290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda2b9c-2fab-4fe0-9cb8-d8467d710f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:36.373007 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda2b9c-2fab-4fe0-9cb8-d8467d710f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:36.374605 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda2b9c-2fab-4fe0-9cb8-d8467d710f49 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:36.385224 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1397/2802] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:36.389319 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7e0fb-b94e-461e-9d8f-f13dc9f47dfe admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:36.394752 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:36.401282 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7e0fb-b94e-461e-9d8f-f13dc9f47dfe admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:36.406092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb265e5-1f78-4877-b913-6fb0044323b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:36.406739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb265e5-1f78-4877-b913-6fb0044323b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:36.407022 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb265e5-1f78-4877-b913-6fb0044323b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:36.407878 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb265e5-1f78-4877-b913-6fb0044323b1 admin admin] PATH_INFO: `/v3/projects/dbf51e9bb72044e5a1b4dcbf0c597ce5/users/abc707a5af9a45ae92c1d8b9e7503d0b/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:36.409844 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb265e5-1f78-4877-b913-6fb0044323b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf51e9bb72044e5a1b4dcbf0c597ce5, user_id=abc707a5af9a45ae92c1d8b9e7503d0b, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:36.412504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb265e5-1f78-4877-b913-6fb0044323b1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:36.422419 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1406/2803] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:49:36 2022] PUT /identity/v3/projects/dbf51e9bb72044e5a1b4dcbf0c597ce5/users/abc707a5af9a45ae92c1d8b9e7503d0b/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:36.434197 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122c938-1211-4483-8fc7-d108d20b4aff None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:36.434690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122c938-1211-4483-8fc7-d108d20b4aff None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:36.434791 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122c938-1211-4483-8fc7-d108d20b4aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:36.461882 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-a122c938-1211-4483-8fc7-d108d20b4aff None None] MFA Rules not processed for user `abc707a5af9a45ae92c1d8b9e7503d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:36.463027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-a122c938-1211-4483-8fc7-d108d20b4aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:36.528987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1398/2804] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:36.541762 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb265e5-1f78-4877-b913-6fb0044323b1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:36.543545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-edb265e5-1f78-4877-b913-6fb0044323b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:36.600367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb265e5-1f78-4877-b913-6fb0044323b1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:36.603934 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1c0d8-cda6-4329-9620-796f7041e1fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:36.604534 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1c0d8-cda6-4329-9620-796f7041e1fa service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:36.604814 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1c0d8-cda6-4329-9620-796f7041e1fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:36.605086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1c0d8-cda6-4329-9620-796f7041e1fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:36.614466 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1c0d8-cda6-4329-9620-796f7041e1fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:36.615917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1c0d8-cda6-4329-9620-796f7041e1fa service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:36.625016 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1407/2805] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:36.933392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122c938-1211-4483-8fc7-d108d20b4aff None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:36.943251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122c938-1211-4483-8fc7-d108d20b4aff None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:36.947880 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702758e4-c5ca-48a9-acbd-3711bb721b23 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc707a5af9a45ae92c1d8b9e7503d0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dbf51e9bb72044e5a1b4dcbf0c597ce5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:36.948502 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702758e4-c5ca-48a9-acbd-3711bb721b23 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:36.948796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702758e4-c5ca-48a9-acbd-3711bb721b23 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:36.949079 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702758e4-c5ca-48a9-acbd-3711bb721b23 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:36.949844 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702758e4-c5ca-48a9-acbd-3711bb721b23 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:49:36.950100 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702758e4-c5ca-48a9-acbd-3711bb721b23 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:36.951734 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702758e4-c5ca-48a9-acbd-3711bb721b23 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:36.967198 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1399/2806] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:49:36.968371 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1c0d8-cda6-4329-9620-796f7041e1fa service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:36.978196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1c0d8-cda6-4329-9620-796f7041e1fa service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:36.982746 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4772d9-93f3-482a-abc6-1529bce9c3a2 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc707a5af9a45ae92c1d8b9e7503d0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dbf51e9bb72044e5a1b4dcbf0c597ce5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:36.983317 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4772d9-93f3-482a-abc6-1529bce9c3a2 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:36.983600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4772d9-93f3-482a-abc6-1529bce9c3a2 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:36.983877 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4772d9-93f3-482a-abc6-1529bce9c3a2 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:36.984538 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4772d9-93f3-482a-abc6-1529bce9c3a2 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:36.986699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4772d9-93f3-482a-abc6-1529bce9c3a2 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:37.012775 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1408/2807] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:49:36 2022] POST /identity/v3/projects => generated 353 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:49:37.017427 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702758e4-c5ca-48a9-acbd-3711bb721b23 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:37.027563 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702758e4-c5ca-48a9-acbd-3711bb721b23 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:37.082812 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5fee6-9849-41ec-9f8a-ff051917b803 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc707a5af9a45ae92c1d8b9e7503d0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dbf51e9bb72044e5a1b4dcbf0c597ce5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:37.083474 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5fee6-9849-41ec-9f8a-ff051917b803 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:37.084297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5fee6-9849-41ec-9f8a-ff051917b803 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:37.084297 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5fee6-9849-41ec-9f8a-ff051917b803 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:37.084697 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5fee6-9849-41ec-9f8a-ff051917b803 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:49:37.085867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5fee6-9849-41ec-9f8a-ff051917b803 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:37.086582 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5fee6-9849-41ec-9f8a-ff051917b803 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:37.100871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1400/2808] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:49:37.136970 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4772d9-93f3-482a-abc6-1529bce9c3a2 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:37.147681 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4772d9-93f3-482a-abc6-1529bce9c3a2 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:37.152193 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780aa878-152f-4b7b-97b3-f51ae496ca68 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc707a5af9a45ae92c1d8b9e7503d0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dbf51e9bb72044e5a1b4dcbf0c597ce5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:37.152823 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780aa878-152f-4b7b-97b3-f51ae496ca68 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:37.153108 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780aa878-152f-4b7b-97b3-f51ae496ca68 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:37.153404 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780aa878-152f-4b7b-97b3-f51ae496ca68 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] PATH_INFO: `/v3/projects/6d5711d722ec4fe581ab04188928e096` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:37.154530 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780aa878-152f-4b7b-97b3-f51ae496ca68 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6d5711d722ec4fe581ab04188928e096)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:37.156439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780aa878-152f-4b7b-97b3-f51ae496ca68 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:37.202780 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-780aa878-152f-4b7b-97b3-f51ae496ca68 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d5711d722ec4fe581ab04188928e096', 'request_id': 'req-780aa878-152f-4b7b-97b3-f51ae496ca68'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:37.210947 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-780aa878-152f-4b7b-97b3-f51ae496ca68 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d5711d722ec4fe581ab04188928e096', 'request_id': 'req-780aa878-152f-4b7b-97b3-f51ae496ca68'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:37.216758 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-780aa878-152f-4b7b-97b3-f51ae496ca68 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d5711d722ec4fe581ab04188928e096', 'request_id': 'req-780aa878-152f-4b7b-97b3-f51ae496ca68'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:37.228028 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-780aa878-152f-4b7b-97b3-f51ae496ca68 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] The token cache is being invalidate because project 6d5711d722ec4fe581ab04188928e096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:49:37.228313 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-780aa878-152f-4b7b-97b3-f51ae496ca68 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:37.228957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-780aa878-152f-4b7b-97b3-f51ae496ca68 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:37.230446 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1409/2809] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:49:37 2022] DELETE /identity/v3/projects/6d5711d722ec4fe581ab04188928e096 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:37.418770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5fee6-9849-41ec-9f8a-ff051917b803 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:37.420623 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-adc5fee6-9849-41ec-9f8a-ff051917b803 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:37.452264 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:37.486351 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5fee6-9849-41ec-9f8a-ff051917b803 tempest-VolumeQuotaClassesTest-1373866798 tempest-VolumeQuotaClassesTest-1373866798-project-admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:37.539871 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:37.540512 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:37.540799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:37.541078 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] PATH_INFO: `/v3/users/dc05c3c30e724d69b419100bdbc28c97` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:37.542306 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc05c3c30e724d69b419100bdbc28c97)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:37.544269 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:37.590138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc05c3c30e724d69b419100bdbc28c97'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:37.601824 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc05c3c30e724d69b419100bdbc28c97'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:37.602478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc05c3c30e724d69b419100bdbc28c97'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:37.609261 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc05c3c30e724d69b419100bdbc28c97'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:37.616636 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc05c3c30e724d69b419100bdbc28c97'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:37.622626 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1401/2810] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:49:37 2022] DELETE /identity/v3/users/dc05c3c30e724d69b419100bdbc28c97 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:37.628166 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c727e-7356-4650-a665-59f80056e6c1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:37.628677 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c727e-7356-4650-a665-59f80056e6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:37.628784 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c727e-7356-4650-a665-59f80056e6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:37.646933 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ce6c727e-7356-4650-a665-59f80056e6c1 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:37.648166 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ce6c727e-7356-4650-a665-59f80056e6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:37.693591 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:37.706754 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1410/2811] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:37.716342 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:37.717737 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:37.775357 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ef2a0-8f7c-4640-af61-4f11f68a488b admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:37.778861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b39c6-dc08-4181-a7f5-2725025b6628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:37.779499 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b39c6-dc08-4181-a7f5-2725025b6628 service neutron] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:37.779776 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b39c6-dc08-4181-a7f5-2725025b6628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:37.780052 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b39c6-dc08-4181-a7f5-2725025b6628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:37.787001 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:37.792116 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b39c6-dc08-4181-a7f5-2725025b6628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:37.793748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b39c6-dc08-4181-a7f5-2725025b6628 service neutron] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:37.805828 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1402/2812] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:38.076723 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c727e-7356-4650-a665-59f80056e6c1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:38.078192 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ce6c727e-7356-4650-a665-59f80056e6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:38.091325 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c727e-7356-4650-a665-59f80056e6c1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:38.096599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:38.097241 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:38.097525 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:38.097807 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] PATH_INFO: `/v3/projects/4af88dc9eb944ae3bd3b3ca8b3ca8546` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:38.098966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af88dc9eb944ae3bd3b3ca8b3ca8546)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:38.100998 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:38.148285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af88dc9eb944ae3bd3b3ca8b3ca8546', 'request_id': 'req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.155418 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af88dc9eb944ae3bd3b3ca8b3ca8546', 'request_id': 'req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.161501 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af88dc9eb944ae3bd3b3ca8b3ca8546', 'request_id': 'req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.172197 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] The token cache is being invalidate because project 4af88dc9eb944ae3bd3b3ca8b3ca8546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:49:38.172483 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.173178 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.174668 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1411/2813] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:49:38 2022] DELETE /identity/v3/projects/4af88dc9eb944ae3bd3b3ca8b3ca8546 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:38.178822 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b39c6-dc08-4181-a7f5-2725025b6628 service neutron] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:38.189023 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b39c6-dc08-4181-a7f5-2725025b6628 service neutron] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:38.244339 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9d5e8-8b53-4d80-bb6d-2bc7f1bc7c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:38.244978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9d5e8-8b53-4d80-bb6d-2bc7f1bc7c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:38.245284 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9d5e8-8b53-4d80-bb6d-2bc7f1bc7c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:38.245547 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9d5e8-8b53-4d80-bb6d-2bc7f1bc7c2a admin admin] PATH_INFO: `/v3/users/abc707a5af9a45ae92c1d8b9e7503d0b` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:38.246710 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9d5e8-8b53-4d80-bb6d-2bc7f1bc7c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc707a5af9a45ae92c1d8b9e7503d0b)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:38.248668 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9d5e8-8b53-4d80-bb6d-2bc7f1bc7c2a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:38.299588 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cbc9d5e8-8b53-4d80-bb6d-2bc7f1bc7c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc707a5af9a45ae92c1d8b9e7503d0b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.311579 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cbc9d5e8-8b53-4d80-bb6d-2bc7f1bc7c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc707a5af9a45ae92c1d8b9e7503d0b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.312238 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cbc9d5e8-8b53-4d80-bb6d-2bc7f1bc7c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc707a5af9a45ae92c1d8b9e7503d0b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.319730 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cbc9d5e8-8b53-4d80-bb6d-2bc7f1bc7c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc707a5af9a45ae92c1d8b9e7503d0b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.326961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-cbc9d5e8-8b53-4d80-bb6d-2bc7f1bc7c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc707a5af9a45ae92c1d8b9e7503d0b'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.333595 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1403/2814] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:49:38 2022] DELETE /identity/v3/users/abc707a5af9a45ae92c1d8b9e7503d0b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:38.608871 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:38.610928 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:38.623889 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad3e8e-ca17-4d79-9709-cfc6cb41251d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:38.681335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d03565-a5b0-4424-84c3-c26650a59c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:38.681982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d03565-a5b0-4424-84c3-c26650a59c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:38.682246 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d03565-a5b0-4424-84c3-c26650a59c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:38.682524 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d03565-a5b0-4424-84c3-c26650a59c2f admin admin] PATH_INFO: `/v3/projects/dbf51e9bb72044e5a1b4dcbf0c597ce5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:38.683698 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d03565-a5b0-4424-84c3-c26650a59c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf51e9bb72044e5a1b4dcbf0c597ce5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:38.685621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d03565-a5b0-4424-84c3-c26650a59c2f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:38.730952 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-89d03565-a5b0-4424-84c3-c26650a59c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf51e9bb72044e5a1b4dcbf0c597ce5', 'request_id': 'req-89d03565-a5b0-4424-84c3-c26650a59c2f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.738906 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-89d03565-a5b0-4424-84c3-c26650a59c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf51e9bb72044e5a1b4dcbf0c597ce5', 'request_id': 'req-89d03565-a5b0-4424-84c3-c26650a59c2f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.744454 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-89d03565-a5b0-4424-84c3-c26650a59c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf51e9bb72044e5a1b4dcbf0c597ce5', 'request_id': 'req-89d03565-a5b0-4424-84c3-c26650a59c2f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.755548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-89d03565-a5b0-4424-84c3-c26650a59c2f admin admin] The token cache is being invalidate because project dbf51e9bb72044e5a1b4dcbf0c597ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:49:38.755840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-89d03565-a5b0-4424-84c3-c26650a59c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.756476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-89d03565-a5b0-4424-84c3-c26650a59c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:38.758003 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1412/2815] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:49:38 2022] DELETE /identity/v3/projects/dbf51e9bb72044e5a1b4dcbf0c597ce5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:38.762985 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00b8e1-55e7-40c7-954a-ef462509364a None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:38.763481 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00b8e1-55e7-40c7-954a-ef462509364a None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:38.763603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00b8e1-55e7-40c7-954a-ef462509364a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:38.783340 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-0e00b8e1-55e7-40c7-954a-ef462509364a None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:38.784549 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-0e00b8e1-55e7-40c7-954a-ef462509364a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:38.842245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1404/2816] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:38.850618 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd73d0-7b09-4811-978a-74f05e61c87b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:38.850909 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd73d0-7b09-4811-978a-74f05e61c87b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:38.851232 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd73d0-7b09-4811-978a-74f05e61c87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:38.866703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:38.872640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-62dd73d0-7b09-4811-978a-74f05e61c87b None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:38.874504 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-62dd73d0-7b09-4811-978a-74f05e61c87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:38.883312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1413/2817] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:38.887559 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00b8e1-55e7-40c7-954a-ef462509364a None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:38.897281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00b8e1-55e7-40c7-954a-ef462509364a None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:38.901861 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886efd19-b7bd-4daa-a3bb-50e39550dd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:38.902463 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886efd19-b7bd-4daa-a3bb-50e39550dd7c admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:38.902751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886efd19-b7bd-4daa-a3bb-50e39550dd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:38.903029 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886efd19-b7bd-4daa-a3bb-50e39550dd7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:38.903752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886efd19-b7bd-4daa-a3bb-50e39550dd7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80159) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:49:38.904064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886efd19-b7bd-4daa-a3bb-50e39550dd7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:38.905662 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886efd19-b7bd-4daa-a3bb-50e39550dd7c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:38.917739 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1405/2818] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:49:38.922114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd73d0-7b09-4811-978a-74f05e61c87b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:38.931878 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd73d0-7b09-4811-978a-74f05e61c87b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:38.936529 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86718407-3f1c-4c15-90e6-3784ea2b3216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:38.937129 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86718407-3f1c-4c15-90e6-3784ea2b3216 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:38.937412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86718407-3f1c-4c15-90e6-3784ea2b3216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:38.937689 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86718407-3f1c-4c15-90e6-3784ea2b3216 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:38.938349 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86718407-3f1c-4c15-90e6-3784ea2b3216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:38.940965 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86718407-3f1c-4c15-90e6-3784ea2b3216 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:38.969485 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1414/2819] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:49:38 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:38.973956 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886efd19-b7bd-4daa-a3bb-50e39550dd7c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:38.979392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:38.985798 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886efd19-b7bd-4daa-a3bb-50e39550dd7c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.042801 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7fa0f-2af8-4b35-b1ca-8335f226ef5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.043480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7fa0f-2af8-4b35-b1ca-8335f226ef5e admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.043770 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7fa0f-2af8-4b35-b1ca-8335f226ef5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.044054 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7fa0f-2af8-4b35-b1ca-8335f226ef5e admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.044725 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7fa0f-2af8-4b35-b1ca-8335f226ef5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.046672 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7fa0f-2af8-4b35-b1ca-8335f226ef5e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.068309 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1406/2820] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:49:38 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:39.072538 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86718407-3f1c-4c15-90e6-3784ea2b3216 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:39.082951 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86718407-3f1c-4c15-90e6-3784ea2b3216 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.087659 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749117f5-f032-4ab0-9ee5-70fcb1a11fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.088320 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749117f5-f032-4ab0-9ee5-70fcb1a11fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.088610 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749117f5-f032-4ab0-9ee5-70fcb1a11fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.088891 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749117f5-f032-4ab0-9ee5-70fcb1a11fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.089613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749117f5-f032-4ab0-9ee5-70fcb1a11fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.091236 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749117f5-f032-4ab0-9ee5-70fcb1a11fb7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.101219 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1415/2821] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:39.105062 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7fa0f-2af8-4b35-b1ca-8335f226ef5e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:39.115555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7fa0f-2af8-4b35-b1ca-8335f226ef5e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.120242 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a82d12-9d91-4dbb-ac80-3134f838a9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.120783 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a82d12-9d91-4dbb-ac80-3134f838a9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.121073 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a82d12-9d91-4dbb-ac80-3134f838a9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.121355 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a82d12-9d91-4dbb-ac80-3134f838a9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.122046 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a82d12-9d91-4dbb-ac80-3134f838a9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.123732 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a82d12-9d91-4dbb-ac80-3134f838a9a2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.135120 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1407/2822] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:39.139250 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749117f5-f032-4ab0-9ee5-70fcb1a11fb7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:39.149121 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749117f5-f032-4ab0-9ee5-70fcb1a11fb7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.153657 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9c230-1694-4638-bd99-887e62df6302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.154300 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9c230-1694-4638-bd99-887e62df6302 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.154581 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9c230-1694-4638-bd99-887e62df6302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.154859 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9c230-1694-4638-bd99-887e62df6302 admin admin] PATH_INFO: `/v3/projects/5672d25dc51f4d51932f45965b4531a7/users/3834d6b4755e4a16ba4004a8ef41afcb/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.169294 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9c230-1694-4638-bd99-887e62df6302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5672d25dc51f4d51932f45965b4531a7, user_id=3834d6b4755e4a16ba4004a8ef41afcb, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.171972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9c230-1694-4638-bd99-887e62df6302 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.184984 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1416/2823] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:49:39 2022] PUT /identity/v3/projects/5672d25dc51f4d51932f45965b4531a7/users/3834d6b4755e4a16ba4004a8ef41afcb/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:49:39.195698 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc1e63-828b-4fe6-971a-8f418ed76219 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.195996 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc1e63-828b-4fe6-971a-8f418ed76219 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.196281 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc1e63-828b-4fe6-971a-8f418ed76219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.223471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6cdc1e63-828b-4fe6-971a-8f418ed76219 None None] MFA Rules not processed for user `3834d6b4755e4a16ba4004a8ef41afcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:39.224747 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6cdc1e63-828b-4fe6-971a-8f418ed76219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:39.290418 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1408/2824] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:39.294825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9c230-1694-4638-bd99-887e62df6302 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:39.355868 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9c230-1694-4638-bd99-887e62df6302 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.361189 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95137780-6d04-4e91-9dec-de397586bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.361921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95137780-6d04-4e91-9dec-de397586bab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.362209 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95137780-6d04-4e91-9dec-de397586bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.362487 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95137780-6d04-4e91-9dec-de397586bab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.363168 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95137780-6d04-4e91-9dec-de397586bab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.365063 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95137780-6d04-4e91-9dec-de397586bab2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.391926 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1417/2825] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:49:39 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:39.395867 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc1e63-828b-4fe6-971a-8f418ed76219 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:39.406405 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc1e63-828b-4fe6-971a-8f418ed76219 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.462352 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8dd69-b419-4e49-bf34-9bc5f0159f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.462967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8dd69-b419-4e49-bf34-9bc5f0159f03 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.463286 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8dd69-b419-4e49-bf34-9bc5f0159f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.463568 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8dd69-b419-4e49-bf34-9bc5f0159f03 admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.464237 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8dd69-b419-4e49-bf34-9bc5f0159f03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.466163 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8dd69-b419-4e49-bf34-9bc5f0159f03 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.482763 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:39.491241 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1409/2826] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:49:39 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:39.492732 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95137780-6d04-4e91-9dec-de397586bab2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:39.503485 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95137780-6d04-4e91-9dec-de397586bab2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.508081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf5da2-9663-49da-ab3b-0b088f2d1e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.508687 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf5da2-9663-49da-ab3b-0b088f2d1e3b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.508972 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf5da2-9663-49da-ab3b-0b088f2d1e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.509252 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf5da2-9663-49da-ab3b-0b088f2d1e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.509941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf5da2-9663-49da-ab3b-0b088f2d1e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.511571 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf5da2-9663-49da-ab3b-0b088f2d1e3b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.522086 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1418/2827] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:39.525853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8dd69-b419-4e49-bf34-9bc5f0159f03 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:39.536307 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8dd69-b419-4e49-bf34-9bc5f0159f03 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.540874 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8da09-4e78-4286-878f-0e8c27289b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.541469 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8da09-4e78-4286-878f-0e8c27289b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.541748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8da09-4e78-4286-878f-0e8c27289b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.542055 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8da09-4e78-4286-878f-0e8c27289b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.542740 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8da09-4e78-4286-878f-0e8c27289b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.544369 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8da09-4e78-4286-878f-0e8c27289b88 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.554825 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1410/2828] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:39.558832 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf5da2-9663-49da-ab3b-0b088f2d1e3b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:39.569310 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf5da2-9663-49da-ab3b-0b088f2d1e3b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.573997 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69d817-698d-45df-a690-e05a2c571a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.574663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69d817-698d-45df-a690-e05a2c571a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.574952 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69d817-698d-45df-a690-e05a2c571a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.575262 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69d817-698d-45df-a690-e05a2c571a54 admin admin] PATH_INFO: `/v3/projects/b8b17435b52e4f89ade4771408a9b255/users/ef1c3a073783482d89738bf497c12c41/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.591642 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69d817-698d-45df-a690-e05a2c571a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8b17435b52e4f89ade4771408a9b255, user_id=ef1c3a073783482d89738bf497c12c41, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.594273 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69d817-698d-45df-a690-e05a2c571a54 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.607760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1419/2829] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:49:39 2022] PUT /identity/v3/projects/b8b17435b52e4f89ade4771408a9b255/users/ef1c3a073783482d89738bf497c12c41/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:49:39.620134 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a0520-8552-4494-90a7-96ce5ecc33eb None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.620505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a0520-8552-4494-90a7-96ce5ecc33eb None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.620853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a0520-8552-4494-90a7-96ce5ecc33eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.648609 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f05a0520-8552-4494-90a7-96ce5ecc33eb None None] MFA Rules not processed for user `ef1c3a073783482d89738bf497c12c41`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:39.649752 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f05a0520-8552-4494-90a7-96ce5ecc33eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:39.714273 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1411/2830] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:39.718801 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69d817-698d-45df-a690-e05a2c571a54 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:39.770850 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:39.781956 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69d817-698d-45df-a690-e05a2c571a54 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.787194 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a02604-5daa-4424-9ea6-b91e2d5b6c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.787810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a02604-5daa-4424-9ea6-b91e2d5b6c22 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.788101 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a02604-5daa-4424-9ea6-b91e2d5b6c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.788527 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a02604-5daa-4424-9ea6-b91e2d5b6c22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.789089 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a02604-5daa-4424-9ea6-b91e2d5b6c22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.790966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a02604-5daa-4424-9ea6-b91e2d5b6c22 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.817714 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1420/2831] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:49:39 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 07:49:39.822033 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a0520-8552-4494-90a7-96ce5ecc33eb None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:39.833319 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a0520-8552-4494-90a7-96ce5ecc33eb None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.888371 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90c2c1-982b-49f8-ae38-89a6ca626b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.888983 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90c2c1-982b-49f8-ae38-89a6ca626b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.889296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90c2c1-982b-49f8-ae38-89a6ca626b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.889573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90c2c1-982b-49f8-ae38-89a6ca626b1a admin admin] PATH_INFO: `/v3/users` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.890239 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90c2c1-982b-49f8-ae38-89a6ca626b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.892729 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90c2c1-982b-49f8-ae38-89a6ca626b1a admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.913967 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1412/2832] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:49:39 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:39.917892 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a02604-5daa-4424-9ea6-b91e2d5b6c22 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:39.928909 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a02604-5daa-4424-9ea6-b91e2d5b6c22 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.933445 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d25b2-b16f-40f9-a205-676d72f4bda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.934049 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d25b2-b16f-40f9-a205-676d72f4bda2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.934331 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d25b2-b16f-40f9-a205-676d72f4bda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.935110 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d25b2-b16f-40f9-a205-676d72f4bda2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.935341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d25b2-b16f-40f9-a205-676d72f4bda2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.936923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d25b2-b16f-40f9-a205-676d72f4bda2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.948296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1421/2833] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:39.952461 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90c2c1-982b-49f8-ae38-89a6ca626b1a admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:39.962689 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90c2c1-982b-49f8-ae38-89a6ca626b1a admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:39.967478 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1f79d-a665-4c24-9075-738c0935825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:39.968135 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1f79d-a665-4c24-9075-738c0935825e admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:39.968417 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1f79d-a665-4c24-9075-738c0935825e admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:39.968694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1f79d-a665-4c24-9075-738c0935825e admin admin] PATH_INFO: `/v3/projects/dd81de06199f45cfbce018a386bc56be/users/43b05ac47d2242c28274d81ae1747eb8/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:39.983597 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1f79d-a665-4c24-9075-738c0935825e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd81de06199f45cfbce018a386bc56be, user_id=43b05ac47d2242c28274d81ae1747eb8, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:39.986786 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1f79d-a665-4c24-9075-738c0935825e admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:39.999536 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1413/2834] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:49:39 2022] PUT /identity/v3/projects/dd81de06199f45cfbce018a386bc56be/users/43b05ac47d2242c28274d81ae1747eb8/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:40.004558 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d25b2-b16f-40f9-a205-676d72f4bda2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:40.014438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d25b2-b16f-40f9-a205-676d72f4bda2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:40.068739 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8245d-dd35-4c63-b090-2f020f17466c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:40.069347 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8245d-dd35-4c63-b090-2f020f17466c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:40.069633 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8245d-dd35-4c63-b090-2f020f17466c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:40.069910 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8245d-dd35-4c63-b090-2f020f17466c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:40.070600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8245d-dd35-4c63-b090-2f020f17466c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:40.072215 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8245d-dd35-4c63-b090-2f020f17466c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:40.081946 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1422/2835] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:40.086443 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1f79d-a665-4c24-9075-738c0935825e admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:40.096841 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1f79d-a665-4c24-9075-738c0935825e admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:40.101398 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e96be-8033-4255-a2c6-5c514ed4e8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:40.102042 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e96be-8033-4255-a2c6-5c514ed4e8ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:40.102323 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e96be-8033-4255-a2c6-5c514ed4e8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:40.102602 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e96be-8033-4255-a2c6-5c514ed4e8ad admin admin] PATH_INFO: `/v3/projects/dd81de06199f45cfbce018a386bc56be/users/43b05ac47d2242c28274d81ae1747eb8/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:40.104555 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e96be-8033-4255-a2c6-5c514ed4e8ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd81de06199f45cfbce018a386bc56be, user_id=43b05ac47d2242c28274d81ae1747eb8, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:40.107202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e96be-8033-4255-a2c6-5c514ed4e8ad admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:40.117994 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1414/2836] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:49:40 2022] PUT /identity/v3/projects/dd81de06199f45cfbce018a386bc56be/users/43b05ac47d2242c28274d81ae1747eb8/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:40.129134 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030f493-b2de-4f64-9f10-3f3d5c7761ad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:40.129427 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030f493-b2de-4f64-9f10-3f3d5c7761ad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:40.129704 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030f493-b2de-4f64-9f10-3f3d5c7761ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:40.156551 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c030f493-b2de-4f64-9f10-3f3d5c7761ad None None] MFA Rules not processed for user `43b05ac47d2242c28274d81ae1747eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:40.158288 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c030f493-b2de-4f64-9f10-3f3d5c7761ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:40.229030 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1423/2837] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:40.239897 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e96be-8033-4255-a2c6-5c514ed4e8ad admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:40.241615 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-5f4e96be-8033-4255-a2c6-5c514ed4e8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:40.278205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:40.303873 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e96be-8033-4255-a2c6-5c514ed4e8ad admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:40.307515 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b2ffd-b8a6-4672-977b-b3a07a17ce67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:40.308161 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b2ffd-b8a6-4672-977b-b3a07a17ce67 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:40.308442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b2ffd-b8a6-4672-977b-b3a07a17ce67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:40.308718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b2ffd-b8a6-4672-977b-b3a07a17ce67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:40.318760 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b2ffd-b8a6-4672-977b-b3a07a17ce67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:40.320229 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b2ffd-b8a6-4672-977b-b3a07a17ce67 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:40.330666 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1415/2838] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:40.823129 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030f493-b2de-4f64-9f10-3f3d5c7761ad None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:40.824555 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c030f493-b2de-4f64-9f10-3f3d5c7761ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:40.836639 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030f493-b2de-4f64-9f10-3f3d5c7761ad None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:40.840196 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69abf2-10ba-4082-b258-b16ce3c4b257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:40.840804 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69abf2-10ba-4082-b258-b16ce3c4b257 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:40.841087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69abf2-10ba-4082-b258-b16ce3c4b257 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:40.841368 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69abf2-10ba-4082-b258-b16ce3c4b257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:40.885930 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:40.898481 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69abf2-10ba-4082-b258-b16ce3c4b257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:40.899958 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69abf2-10ba-4082-b258-b16ce3c4b257 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:40.909301 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1424/2839] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:44.901811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b2ffd-b8a6-4672-977b-b3a07a17ce67 service cinder] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:44.910635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b2ffd-b8a6-4672-977b-b3a07a17ce67 service cinder] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:44.914185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7ca5d-d7eb-4a80-b25b-9043059beace service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:44.914784 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7ca5d-d7eb-4a80-b25b-9043059beace service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:44.915082 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7ca5d-d7eb-4a80-b25b-9043059beace service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:44.915363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7ca5d-d7eb-4a80-b25b-9043059beace service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:44.971981 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7ca5d-d7eb-4a80-b25b-9043059beace service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:44.973414 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7ca5d-d7eb-4a80-b25b-9043059beace service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:44.982227 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1416/2840] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:47.063274 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69abf2-10ba-4082-b258-b16ce3c4b257 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:47.124813 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69abf2-10ba-4082-b258-b16ce3c4b257 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:47.130438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:47.131083 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:47.131398 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:47.131679 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] PATH_INFO: `/v3/users/3834d6b4755e4a16ba4004a8ef41afcb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:47.132882 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3834d6b4755e4a16ba4004a8ef41afcb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:47.134806 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:47.178833 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3834d6b4755e4a16ba4004a8ef41afcb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.185195 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3834d6b4755e4a16ba4004a8ef41afcb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.197216 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3834d6b4755e4a16ba4004a8ef41afcb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.197840 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3834d6b4755e4a16ba4004a8ef41afcb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.198457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3834d6b4755e4a16ba4004a8ef41afcb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.211329 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1425/2841] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:49:47 2022] DELETE /identity/v3/users/3834d6b4755e4a16ba4004a8ef41afcb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:47.215820 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f9bd9-afbf-4d44-a279-9cb0ed6a1af6 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:47.216308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f9bd9-afbf-4d44-a279-9cb0ed6a1af6 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:47.216308 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f9bd9-afbf-4d44-a279-9cb0ed6a1af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:47.235879 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-b71f9bd9-afbf-4d44-a279-9cb0ed6a1af6 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:47.237644 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b71f9bd9-afbf-4d44-a279-9cb0ed6a1af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:47.299093 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1417/2842] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:47.309457 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:47.310873 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:47.368696 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5a119-915f-47d8-b736-9a23b610d226 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:47.372412 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98569cd-edec-485c-8723-f6954ecac389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:47.373029 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98569cd-edec-485c-8723-f6954ecac389 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:47.373321 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98569cd-edec-485c-8723-f6954ecac389 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:47.373600 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98569cd-edec-485c-8723-f6954ecac389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:47.383932 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98569cd-edec-485c-8723-f6954ecac389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:47.385362 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98569cd-edec-485c-8723-f6954ecac389 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:47.394744 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1426/2843] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:47.670456 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f9bd9-afbf-4d44-a279-9cb0ed6a1af6 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:47.672973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-b71f9bd9-afbf-4d44-a279-9cb0ed6a1af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:47.690901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f9bd9-afbf-4d44-a279-9cb0ed6a1af6 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:47.695438 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:47.696080 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:47.696364 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:47.697250 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] PATH_INFO: `/v3/projects/5672d25dc51f4d51932f45965b4531a7` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:47.698571 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5672d25dc51f4d51932f45965b4531a7)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:47.700489 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:47.748067 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5672d25dc51f4d51932f45965b4531a7', 'request_id': 'req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.754024 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5672d25dc51f4d51932f45965b4531a7', 'request_id': 'req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.760246 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5672d25dc51f4d51932f45965b4531a7', 'request_id': 'req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.763845 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:47.774943 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] The token cache is being invalidate because project 5672d25dc51f4d51932f45965b4531a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:49:47.775278 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.775898 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.777384 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1418/2844] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:49:47 2022] DELETE /identity/v3/projects/5672d25dc51f4d51932f45965b4531a7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:47.781315 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98569cd-edec-485c-8723-f6954ecac389 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:47.791613 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98569cd-edec-485c-8723-f6954ecac389 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:47.796139 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:47.849982 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:47.850621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:47.850907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:47.851222 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] PATH_INFO: `/v3/users/ef1c3a073783482d89738bf497c12c41` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:47.852384 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1c3a073783482d89738bf497c12c41)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:47.854971 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:47.902885 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1c3a073783482d89738bf497c12c41'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.910562 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1c3a073783482d89738bf497c12c41'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.922358 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1c3a073783482d89738bf497c12c41'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.923000 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1c3a073783482d89738bf497c12c41'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.923643 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1c3a073783482d89738bf497c12c41'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:47.936652 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1427/2845] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:49:47 2022] DELETE /identity/v3/users/ef1c3a073783482d89738bf497c12c41 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:48.208122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:48.209766 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:48.222828 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbbaad-c87f-4f29-91c6-2bc8d57972d8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:48.282811 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:48.283500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:48.283782 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:48.284089 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] PATH_INFO: `/v3/projects/b8b17435b52e4f89ade4771408a9b255` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:48.285259 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8b17435b52e4f89ade4771408a9b255)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:48.287199 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:48.333061 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8b17435b52e4f89ade4771408a9b255', 'request_id': 'req-4654a0ba-e195-4e15-9e3b-e52603ea5db0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.338505 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8b17435b52e4f89ade4771408a9b255', 'request_id': 'req-4654a0ba-e195-4e15-9e3b-e52603ea5db0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.344558 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8b17435b52e4f89ade4771408a9b255', 'request_id': 'req-4654a0ba-e195-4e15-9e3b-e52603ea5db0'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.357533 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] The token cache is being invalidate because project b8b17435b52e4f89ade4771408a9b255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:49:48.357824 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.358442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.359989 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1419/2846] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:49:48 2022] DELETE /identity/v3/projects/b8b17435b52e4f89ade4771408a9b255 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:48.364825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:48.366359 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:48.380438 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbfaba-8741-444f-9a74-9b6ed6972f4e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:48.442957 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4be17-b570-44c1-b593-c6f0a1975692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:48.443625 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4be17-b570-44c1-b593-c6f0a1975692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:48.443911 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4be17-b570-44c1-b593-c6f0a1975692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:48.444190 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4be17-b570-44c1-b593-c6f0a1975692 admin admin] PATH_INFO: `/v3/users/43b05ac47d2242c28274d81ae1747eb8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:48.445350 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4be17-b570-44c1-b593-c6f0a1975692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b05ac47d2242c28274d81ae1747eb8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:48.447299 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4be17-b570-44c1-b593-c6f0a1975692 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:48.479886 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:48.494494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-55a4be17-b570-44c1-b593-c6f0a1975692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b05ac47d2242c28274d81ae1747eb8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.501640 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-55a4be17-b570-44c1-b593-c6f0a1975692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b05ac47d2242c28274d81ae1747eb8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.513825 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-55a4be17-b570-44c1-b593-c6f0a1975692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b05ac47d2242c28274d81ae1747eb8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.514464 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-55a4be17-b570-44c1-b593-c6f0a1975692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b05ac47d2242c28274d81ae1747eb8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.515092 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-55a4be17-b570-44c1-b593-c6f0a1975692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b05ac47d2242c28274d81ae1747eb8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.526855 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1428/2847] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:49:48 2022] DELETE /identity/v3/users/43b05ac47d2242c28274d81ae1747eb8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:48.798751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:48.800767 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:48.813830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654a0ba-e195-4e15-9e3b-e52603ea5db0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:48.872069 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74881f-72b8-4cdc-80c1-bdc9c142b492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:48.872718 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74881f-72b8-4cdc-80c1-bdc9c142b492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:48.873003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74881f-72b8-4cdc-80c1-bdc9c142b492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:48.873287 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74881f-72b8-4cdc-80c1-bdc9c142b492 admin admin] PATH_INFO: `/v3/projects/dd81de06199f45cfbce018a386bc56be` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:48.874466 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74881f-72b8-4cdc-80c1-bdc9c142b492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd81de06199f45cfbce018a386bc56be)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:48.876402 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74881f-72b8-4cdc-80c1-bdc9c142b492 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:48.907338 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:48.923830 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8b74881f-72b8-4cdc-80c1-bdc9c142b492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd81de06199f45cfbce018a386bc56be', 'request_id': 'req-8b74881f-72b8-4cdc-80c1-bdc9c142b492'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.929699 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8b74881f-72b8-4cdc-80c1-bdc9c142b492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd81de06199f45cfbce018a386bc56be', 'request_id': 'req-8b74881f-72b8-4cdc-80c1-bdc9c142b492'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.935924 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8b74881f-72b8-4cdc-80c1-bdc9c142b492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd81de06199f45cfbce018a386bc56be', 'request_id': 'req-8b74881f-72b8-4cdc-80c1-bdc9c142b492'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.953087 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8b74881f-72b8-4cdc-80c1-bdc9c142b492 admin admin] The token cache is being invalidate because project dd81de06199f45cfbce018a386bc56be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:49:48.953387 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8b74881f-72b8-4cdc-80c1-bdc9c142b492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.954067 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-8b74881f-72b8-4cdc-80c1-bdc9c142b492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:48.955612 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1420/2848] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:49:48 2022] DELETE /identity/v3/projects/dd81de06199f45cfbce018a386bc56be => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:48.962128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6995f01-a03c-4606-870d-1ca8c69340ee None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:48.962519 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6995f01-a03c-4606-870d-1ca8c69340ee None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:48.962866 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6995f01-a03c-4606-870d-1ca8c69340ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:48.982827 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d6995f01-a03c-4606-870d-1ca8c69340ee None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:48.984101 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d6995f01-a03c-4606-870d-1ca8c69340ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:49.043087 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1429/2849] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:49.053392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323a179-d057-4345-ab62-59025ff077ce None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.053613 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323a179-d057-4345-ab62-59025ff077ce None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.053891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323a179-d057-4345-ab62-59025ff077ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.072948 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-1323a179-d057-4345-ab62-59025ff077ce None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:49.074137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-1323a179-d057-4345-ab62-59025ff077ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:49.082562 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1421/2850] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:49.086634 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6995f01-a03c-4606-870d-1ca8c69340ee None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:49.096277 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6995f01-a03c-4606-870d-1ca8c69340ee None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:49.101459 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de152623-89ce-4807-ba03-56f090525cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:49.102063 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de152623-89ce-4807-ba03-56f090525cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.102352 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de152623-89ce-4807-ba03-56f090525cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.102627 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de152623-89ce-4807-ba03-56f090525cb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.103378 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de152623-89ce-4807-ba03-56f090525cb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:49:49.103706 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de152623-89ce-4807-ba03-56f090525cb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:49.105271 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de152623-89ce-4807-ba03-56f090525cb1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:49.116743 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1430/2851] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:49:49.120707 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323a179-d057-4345-ab62-59025ff077ce None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:49.130724 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323a179-d057-4345-ab62-59025ff077ce None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:49.135343 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fa7da-4b20-48d6-9c51-f47f6c0be163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:49.135938 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fa7da-4b20-48d6-9c51-f47f6c0be163 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.136220 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fa7da-4b20-48d6-9c51-f47f6c0be163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.136494 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fa7da-4b20-48d6-9c51-f47f6c0be163 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.137146 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fa7da-4b20-48d6-9c51-f47f6c0be163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:49.139020 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fa7da-4b20-48d6-9c51-f47f6c0be163 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:49.165667 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1422/2852] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:49:49 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:49.169740 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de152623-89ce-4807-ba03-56f090525cb1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:49.179760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de152623-89ce-4807-ba03-56f090525cb1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:49.235826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe077fa-885d-4d05-ada6-e5f304c463e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:49.236341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe077fa-885d-4d05-ada6-e5f304c463e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.236623 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe077fa-885d-4d05-ada6-e5f304c463e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.236898 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe077fa-885d-4d05-ada6-e5f304c463e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.238134 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe077fa-885d-4d05-ada6-e5f304c463e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:49.240070 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe077fa-885d-4d05-ada6-e5f304c463e6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:49.261296 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1431/2853] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:49:49 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:49.265232 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fa7da-4b20-48d6-9c51-f47f6c0be163 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:49.275676 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fa7da-4b20-48d6-9c51-f47f6c0be163 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:49.280179 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e9486-eccb-4945-a0fb-f7c986bca5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:49.280775 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e9486-eccb-4945-a0fb-f7c986bca5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.281056 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e9486-eccb-4945-a0fb-f7c986bca5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.281334 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e9486-eccb-4945-a0fb-f7c986bca5b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.282013 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e9486-eccb-4945-a0fb-f7c986bca5b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:49.283635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e9486-eccb-4945-a0fb-f7c986bca5b0 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:49.294003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1423/2854] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:49.297944 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe077fa-885d-4d05-ada6-e5f304c463e6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:49.308702 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe077fa-885d-4d05-ada6-e5f304c463e6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:49.313187 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da9f75-7bcb-4ded-8e4d-4a0db9e57aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:49.313786 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da9f75-7bcb-4ded-8e4d-4a0db9e57aab admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.314068 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da9f75-7bcb-4ded-8e4d-4a0db9e57aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.314354 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da9f75-7bcb-4ded-8e4d-4a0db9e57aab admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.315090 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da9f75-7bcb-4ded-8e4d-4a0db9e57aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:49.316734 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da9f75-7bcb-4ded-8e4d-4a0db9e57aab admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:49.327199 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1432/2855] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:49.332070 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e9486-eccb-4945-a0fb-f7c986bca5b0 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:49.341933 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e9486-eccb-4945-a0fb-f7c986bca5b0 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:49.346984 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412110d5-d81d-4b9c-b370-86ecbd1d78bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:49.347799 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412110d5-d81d-4b9c-b370-86ecbd1d78bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.348084 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412110d5-d81d-4b9c-b370-86ecbd1d78bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.348363 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412110d5-d81d-4b9c-b370-86ecbd1d78bd admin admin] PATH_INFO: `/v3/projects/e990bfd71c5241c682bb20b22f7a121c/users/e74863cba7a04e72b801826482d2fb7c/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.364004 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412110d5-d81d-4b9c-b370-86ecbd1d78bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e990bfd71c5241c682bb20b22f7a121c, user_id=e74863cba7a04e72b801826482d2fb7c, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:49.366627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412110d5-d81d-4b9c-b370-86ecbd1d78bd admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:49.379256 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1424/2856] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:49:49 2022] PUT /identity/v3/projects/e990bfd71c5241c682bb20b22f7a121c/users/e74863cba7a04e72b801826482d2fb7c/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:49.390547 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894f2b3-d53b-49d1-8913-74373624729c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.391024 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894f2b3-d53b-49d1-8913-74373624729c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.391157 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894f2b3-d53b-49d1-8913-74373624729c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.415803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:49.419224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1894f2b3-d53b-49d1-8913-74373624729c None None] MFA Rules not processed for user `e74863cba7a04e72b801826482d2fb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:49.421056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1894f2b3-d53b-49d1-8913-74373624729c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:49.485055 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1433/2857] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:49:49.490129 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412110d5-d81d-4b9c-b370-86ecbd1d78bd admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:49.552954 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412110d5-d81d-4b9c-b370-86ecbd1d78bd admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:49.557544 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32503bab-2af6-4bee-98cd-2345a6f867e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:49.558148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32503bab-2af6-4bee-98cd-2345a6f867e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.558434 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32503bab-2af6-4bee-98cd-2345a6f867e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.558757 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32503bab-2af6-4bee-98cd-2345a6f867e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.559404 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32503bab-2af6-4bee-98cd-2345a6f867e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:49.561301 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32503bab-2af6-4bee-98cd-2345a6f867e8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:49.588460 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1425/2858] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:49:49 2022] POST /identity/v3/projects => generated 391 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:49.592609 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894f2b3-d53b-49d1-8913-74373624729c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:49.603584 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894f2b3-d53b-49d1-8913-74373624729c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:49.658731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7222b-58ab-4440-8f92-0db482cea97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:49.659376 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7222b-58ab-4440-8f92-0db482cea97f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.659663 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7222b-58ab-4440-8f92-0db482cea97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.659945 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7222b-58ab-4440-8f92-0db482cea97f admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.660615 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e7222b-58ab-4440-8f92-0db482cea97f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:49.662533 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e7222b-58ab-4440-8f92-0db482cea97f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:49.683327 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1434/2859] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:49:49 2022] POST /identity/v3/users => generated 302 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:49.687482 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32503bab-2af6-4bee-98cd-2345a6f867e8 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:49.698188 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32503bab-2af6-4bee-98cd-2345a6f867e8 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:49.702700 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f8d12-e6f2-47e4-8c78-e4dd9aff6c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:49.703327 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f8d12-e6f2-47e4-8c78-e4dd9aff6c13 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.703608 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f8d12-e6f2-47e4-8c78-e4dd9aff6c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.703883 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f8d12-e6f2-47e4-8c78-e4dd9aff6c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.704565 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f8d12-e6f2-47e4-8c78-e4dd9aff6c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:49.706137 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f8d12-e6f2-47e4-8c78-e4dd9aff6c13 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:49.716805 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1426/2860] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:49.720810 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7222b-58ab-4440-8f92-0db482cea97f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:49.730835 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7222b-58ab-4440-8f92-0db482cea97f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:49.735499 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe61ef-6942-4f4e-9691-2c20c20695e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:49.736149 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe61ef-6942-4f4e-9691-2c20c20695e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.736434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe61ef-6942-4f4e-9691-2c20c20695e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.736712 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe61ef-6942-4f4e-9691-2c20c20695e4 admin admin] PATH_INFO: `/v3/projects/f8e5970ea2ca4bd0b3ab6452c1122e0e/users/5762a95bf5ee45db93ee97b1eee6c226/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.751612 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe61ef-6942-4f4e-9691-2c20c20695e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8e5970ea2ca4bd0b3ab6452c1122e0e, user_id=5762a95bf5ee45db93ee97b1eee6c226, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:49.755225 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe61ef-6942-4f4e-9691-2c20c20695e4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:49.767703 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1435/2861] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:49:49 2022] PUT /identity/v3/projects/f8e5970ea2ca4bd0b3ab6452c1122e0e/users/5762a95bf5ee45db93ee97b1eee6c226/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:49.771703 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f8d12-e6f2-47e4-8c78-e4dd9aff6c13 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:49.781500 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f8d12-e6f2-47e4-8c78-e4dd9aff6c13 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:49.805523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:49.840917 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74480f-2a7a-4647-9a03-58ffc0b5fb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:49.841471 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74480f-2a7a-4647-9a03-58ffc0b5fb4b admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.841756 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74480f-2a7a-4647-9a03-58ffc0b5fb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.842035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74480f-2a7a-4647-9a03-58ffc0b5fb4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.842722 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74480f-2a7a-4647-9a03-58ffc0b5fb4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:49.844351 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74480f-2a7a-4647-9a03-58ffc0b5fb4b admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:49.854415 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1427/2862] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:49:49.858536 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe61ef-6942-4f4e-9691-2c20c20695e4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:49.869518 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe61ef-6942-4f4e-9691-2c20c20695e4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:49.876937 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769480df-5ccd-4b8e-b0e8-96ef22932102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:49.878943 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769480df-5ccd-4b8e-b0e8-96ef22932102 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.879505 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769480df-5ccd-4b8e-b0e8-96ef22932102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.880013 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769480df-5ccd-4b8e-b0e8-96ef22932102 admin admin] PATH_INFO: `/v3/projects/f8e5970ea2ca4bd0b3ab6452c1122e0e/users/5762a95bf5ee45db93ee97b1eee6c226/roles/69aa06056979475d8b9fe3bcd9298f02` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.883548 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769480df-5ccd-4b8e-b0e8-96ef22932102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8e5970ea2ca4bd0b3ab6452c1122e0e, user_id=5762a95bf5ee45db93ee97b1eee6c226, role_id=69aa06056979475d8b9fe3bcd9298f02)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:49.886742 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769480df-5ccd-4b8e-b0e8-96ef22932102 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:49.897271 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1436/2863] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:49:49 2022] PUT /identity/v3/projects/f8e5970ea2ca4bd0b3ab6452c1122e0e/users/5762a95bf5ee45db93ee97b1eee6c226/roles/69aa06056979475d8b9fe3bcd9298f02 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:49.907495 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed449a00-f679-42c3-98ad-624fa2263c49 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:49.907796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed449a00-f679-42c3-98ad-624fa2263c49 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:49.908065 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed449a00-f679-42c3-98ad-624fa2263c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:49.936011 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-ed449a00-f679-42c3-98ad-624fa2263c49 None None] MFA Rules not processed for user `5762a95bf5ee45db93ee97b1eee6c226`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:49.937228 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ed449a00-f679-42c3-98ad-624fa2263c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:50.006441 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1428/2864] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:50.017266 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769480df-5ccd-4b8e-b0e8-96ef22932102 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:50.019622 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-769480df-5ccd-4b8e-b0e8-96ef22932102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:50.076260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769480df-5ccd-4b8e-b0e8-96ef22932102 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:50.079875 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e996193-6dbc-4abf-9b2c-306f5f8b25c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:50.080475 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e996193-6dbc-4abf-9b2c-306f5f8b25c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:50.080803 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e996193-6dbc-4abf-9b2c-306f5f8b25c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:50.081107 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e996193-6dbc-4abf-9b2c-306f5f8b25c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:50.089862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e996193-6dbc-4abf-9b2c-306f5f8b25c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:50.091367 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e996193-6dbc-4abf-9b2c-306f5f8b25c4 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:50.095172 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:50.102156 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1437/2865] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:50.147170 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed449a00-f679-42c3-98ad-624fa2263c49 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:50.148542 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-ed449a00-f679-42c3-98ad-624fa2263c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:50.160934 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed449a00-f679-42c3-98ad-624fa2263c49 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:50.164792 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69cb5a-9ba8-4512-8910-f794783060c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:50.165405 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69cb5a-9ba8-4512-8910-f794783060c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:50.165690 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69cb5a-9ba8-4512-8910-f794783060c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:50.165997 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69cb5a-9ba8-4512-8910-f794783060c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:50.222694 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69cb5a-9ba8-4512-8910-f794783060c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:50.224205 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69cb5a-9ba8-4512-8910-f794783060c4 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:50.233003 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1429/2866] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:53.500834 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e996193-6dbc-4abf-9b2c-306f5f8b25c4 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:53.562862 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e996193-6dbc-4abf-9b2c-306f5f8b25c4 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:53.567779 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:53.568413 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:53.568693 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:53.568966 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] PATH_INFO: `/v3/users/e74863cba7a04e72b801826482d2fb7c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:53.570130 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e74863cba7a04e72b801826482d2fb7c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:53.572077 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:53.617695 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e74863cba7a04e72b801826482d2fb7c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:53.624546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e74863cba7a04e72b801826482d2fb7c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:53.636896 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e74863cba7a04e72b801826482d2fb7c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:53.637541 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e74863cba7a04e72b801826482d2fb7c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:53.638150 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e74863cba7a04e72b801826482d2fb7c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:53.650683 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1438/2867] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:49:53 2022] DELETE /identity/v3/users/e74863cba7a04e72b801826482d2fb7c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:53.655187 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922d1c6-1f9c-44cb-8754-111d7ccdb602 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:53.655670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922d1c6-1f9c-44cb-8754-111d7ccdb602 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:53.655779 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922d1c6-1f9c-44cb-8754-111d7ccdb602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:53.676219 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-f922d1c6-1f9c-44cb-8754-111d7ccdb602 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:53.677425 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f922d1c6-1f9c-44cb-8754-111d7ccdb602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:53.718202 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=100 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:53.736317 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1430/2868] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:53.746811 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:53.748326 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:53.813765 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a7f0e-ad3f-4ca7-92e5-df01df15cd19 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:53.817955 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229c58b-ed6b-453f-a315-0a5c89c17d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:53.818565 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229c58b-ed6b-453f-a315-0a5c89c17d93 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:53.818845 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229c58b-ed6b-453f-a315-0a5c89c17d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:53.819148 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229c58b-ed6b-453f-a315-0a5c89c17d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:53.828037 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229c58b-ed6b-453f-a315-0a5c89c17d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:53.829455 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229c58b-ed6b-453f-a315-0a5c89c17d93 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:53.838781 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1439/2869] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:54.108347 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922d1c6-1f9c-44cb-8754-111d7ccdb602 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:54.109855 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-f922d1c6-1f9c-44cb-8754-111d7ccdb602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:54.121939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922d1c6-1f9c-44cb-8754-111d7ccdb602 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:54.126681 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:54.127348 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:54.127635 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:54.128245 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] PATH_INFO: `/v3/projects/e990bfd71c5241c682bb20b22f7a121c` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:54.129751 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e990bfd71c5241c682bb20b22f7a121c)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:54.131831 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:54.178663 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e990bfd71c5241c682bb20b22f7a121c', 'request_id': 'req-9d4ca686-5ba7-443c-8edf-11f45512f9d5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.184185 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e990bfd71c5241c682bb20b22f7a121c', 'request_id': 'req-9d4ca686-5ba7-443c-8edf-11f45512f9d5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.190195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e990bfd71c5241c682bb20b22f7a121c', 'request_id': 'req-9d4ca686-5ba7-443c-8edf-11f45512f9d5'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.202010 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] The token cache is being invalidate because project e990bfd71c5241c682bb20b22f7a121c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:49:54.202296 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.202908 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.204423 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1431/2870] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:49:54 2022] DELETE /identity/v3/projects/e990bfd71c5241c682bb20b22f7a121c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:54.209012 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229c58b-ed6b-453f-a315-0a5c89c17d93 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:54.218460 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229c58b-ed6b-453f-a315-0a5c89c17d93 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:54.234727 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:54.275629 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523f2a9-2144-473a-a749-8827ba06c000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:54.276260 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523f2a9-2144-473a-a749-8827ba06c000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:54.276546 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523f2a9-2144-473a-a749-8827ba06c000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:54.276826 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523f2a9-2144-473a-a749-8827ba06c000 admin admin] PATH_INFO: `/v3/users/5762a95bf5ee45db93ee97b1eee6c226` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:54.277974 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523f2a9-2144-473a-a749-8827ba06c000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5762a95bf5ee45db93ee97b1eee6c226)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:54.279907 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523f2a9-2144-473a-a749-8827ba06c000 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:54.324385 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7523f2a9-2144-473a-a749-8827ba06c000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5762a95bf5ee45db93ee97b1eee6c226'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.331923 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7523f2a9-2144-473a-a749-8827ba06c000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5762a95bf5ee45db93ee97b1eee6c226'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.344173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7523f2a9-2144-473a-a749-8827ba06c000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5762a95bf5ee45db93ee97b1eee6c226'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.344824 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7523f2a9-2144-473a-a749-8827ba06c000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5762a95bf5ee45db93ee97b1eee6c226'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.345428 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7523f2a9-2144-473a-a749-8827ba06c000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5762a95bf5ee45db93ee97b1eee6c226'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.357899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1440/2871] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:49:54 2022] DELETE /identity/v3/users/5762a95bf5ee45db93ee97b1eee6c226 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:54.628961 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:54.630437 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:54.644566 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ca686-5ba7-443c-8edf-11f45512f9d5 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:54.701143 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99856f-f769-45f3-8f3f-9ecbb09e5318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:54.701813 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99856f-f769-45f3-8f3f-9ecbb09e5318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:54.702064 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99856f-f769-45f3-8f3f-9ecbb09e5318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:54.702345 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99856f-f769-45f3-8f3f-9ecbb09e5318 admin admin] PATH_INFO: `/v3/projects/f8e5970ea2ca4bd0b3ab6452c1122e0e` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:54.703642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99856f-f769-45f3-8f3f-9ecbb09e5318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8e5970ea2ca4bd0b3ab6452c1122e0e)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:54.705526 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99856f-f769-45f3-8f3f-9ecbb09e5318 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:54.752789 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5b99856f-f769-45f3-8f3f-9ecbb09e5318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8e5970ea2ca4bd0b3ab6452c1122e0e', 'request_id': 'req-5b99856f-f769-45f3-8f3f-9ecbb09e5318'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.759321 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5b99856f-f769-45f3-8f3f-9ecbb09e5318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8e5970ea2ca4bd0b3ab6452c1122e0e', 'request_id': 'req-5b99856f-f769-45f3-8f3f-9ecbb09e5318'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.765454 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5b99856f-f769-45f3-8f3f-9ecbb09e5318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8e5970ea2ca4bd0b3ab6452c1122e0e', 'request_id': 'req-5b99856f-f769-45f3-8f3f-9ecbb09e5318'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.774218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:54.778392 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5b99856f-f769-45f3-8f3f-9ecbb09e5318 admin admin] The token cache is being invalidate because project f8e5970ea2ca4bd0b3ab6452c1122e0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:49:54.778856 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5b99856f-f769-45f3-8f3f-9ecbb09e5318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.780044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-5b99856f-f769-45f3-8f3f-9ecbb09e5318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:54.781545 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1432/2872] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:49:54 2022] DELETE /identity/v3/projects/f8e5970ea2ca4bd0b3ab6452c1122e0e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:54.786424 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663c0e1-5f67-4947-9e87-8cddf609ded1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:54.786908 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663c0e1-5f67-4947-9e87-8cddf609ded1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:54.787018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663c0e1-5f67-4947-9e87-8cddf609ded1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:54.807054 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1663c0e1-5f67-4947-9e87-8cddf609ded1 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:54.808293 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1663c0e1-5f67-4947-9e87-8cddf609ded1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:54.866114 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1441/2873] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:54.875059 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06a7af-27ab-457a-84f7-8d058306be59 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:54.875388 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06a7af-27ab-457a-84f7-8d058306be59 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:54.875670 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06a7af-27ab-457a-84f7-8d058306be59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:54.896688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-8e06a7af-27ab-457a-84f7-8d058306be59 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:54.897891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-8e06a7af-27ab-457a-84f7-8d058306be59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:54.905675 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1433/2874] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:54.910056 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663c0e1-5f67-4947-9e87-8cddf609ded1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:54.919702 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663c0e1-5f67-4947-9e87-8cddf609ded1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:54.924212 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea16b9-601b-4b47-aeb1-878b4afce46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:54.925605 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea16b9-601b-4b47-aeb1-878b4afce46f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:54.925893 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea16b9-601b-4b47-aeb1-878b4afce46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:54.926173 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea16b9-601b-4b47-aeb1-878b4afce46f admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:54.926887 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea16b9-601b-4b47-aeb1-878b4afce46f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:49:54.927224 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea16b9-601b-4b47-aeb1-878b4afce46f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:54.928817 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea16b9-601b-4b47-aeb1-878b4afce46f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:54.940210 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1442/2875] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 07:49:54.944958 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06a7af-27ab-457a-84f7-8d058306be59 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:54.954302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06a7af-27ab-457a-84f7-8d058306be59 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:54.958911 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ffde4-e9a3-4466-9327-d3c5f45e2d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:54.959551 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ffde4-e9a3-4466-9327-d3c5f45e2d4c admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:54.959833 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ffde4-e9a3-4466-9327-d3c5f45e2d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:54.960106 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ffde4-e9a3-4466-9327-d3c5f45e2d4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:54.960768 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ffde4-e9a3-4466-9327-d3c5f45e2d4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:54.962650 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ffde4-e9a3-4466-9327-d3c5f45e2d4c admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:54.989736 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1434/2876] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:49:54 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:54.993884 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea16b9-601b-4b47-aeb1-878b4afce46f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:55.004660 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea16b9-601b-4b47-aeb1-878b4afce46f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:55.028447 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:55.061877 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac32c3f-c6a4-4283-801b-a70eadd3c477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:55.062496 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac32c3f-c6a4-4283-801b-a70eadd3c477 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:55.062783 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac32c3f-c6a4-4283-801b-a70eadd3c477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:55.063078 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac32c3f-c6a4-4283-801b-a70eadd3c477 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:55.063760 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac32c3f-c6a4-4283-801b-a70eadd3c477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:55.066335 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac32c3f-c6a4-4283-801b-a70eadd3c477 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:55.086751 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1443/2877] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:49:54 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:55.090833 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ffde4-e9a3-4466-9327-d3c5f45e2d4c admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:55.100998 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ffde4-e9a3-4466-9327-d3c5f45e2d4c admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:55.105523 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fbb77-0592-451d-87b1-1365fcf814d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:55.106130 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fbb77-0592-451d-87b1-1365fcf814d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:55.106410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fbb77-0592-451d-87b1-1365fcf814d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:55.106688 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fbb77-0592-451d-87b1-1365fcf814d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:55.107442 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fbb77-0592-451d-87b1-1365fcf814d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:55.109039 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fbb77-0592-451d-87b1-1365fcf814d2 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:55.119992 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1435/2878] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:55.123593 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac32c3f-c6a4-4283-801b-a70eadd3c477 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:55.133989 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac32c3f-c6a4-4283-801b-a70eadd3c477 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:55.138596 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239f4dd-0178-4437-85c0-5ca9d5809518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:55.139225 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239f4dd-0178-4437-85c0-5ca9d5809518 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:55.139510 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239f4dd-0178-4437-85c0-5ca9d5809518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:55.139789 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239f4dd-0178-4437-85c0-5ca9d5809518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:55.140478 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239f4dd-0178-4437-85c0-5ca9d5809518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:55.142058 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239f4dd-0178-4437-85c0-5ca9d5809518 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:55.152494 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1444/2879] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:55.156601 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fbb77-0592-451d-87b1-1365fcf814d2 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:55.167251 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fbb77-0592-451d-87b1-1365fcf814d2 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:55.171990 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ea77d-f090-4611-a4e9-361f9ff2e2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:55.172657 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ea77d-f090-4611-a4e9-361f9ff2e2cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:55.172939 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ea77d-f090-4611-a4e9-361f9ff2e2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:55.173218 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ea77d-f090-4611-a4e9-361f9ff2e2cb admin admin] PATH_INFO: `/v3/projects/068e70a4c3484902aba1ccefe590af73/users/c2b2bc070ec1473d87e36e4bf497a46f/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:55.188772 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ea77d-f090-4611-a4e9-361f9ff2e2cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=068e70a4c3484902aba1ccefe590af73, user_id=c2b2bc070ec1473d87e36e4bf497a46f, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:55.191818 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ea77d-f090-4611-a4e9-361f9ff2e2cb admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:55.204600 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1436/2880] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:49:55 2022] PUT /identity/v3/projects/068e70a4c3484902aba1ccefe590af73/users/c2b2bc070ec1473d87e36e4bf497a46f/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:55.215081 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b6cda-9298-42d7-b0ba-a4cc6acd2d01 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:55.215381 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b6cda-9298-42d7-b0ba-a4cc6acd2d01 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:55.215674 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b6cda-9298-42d7-b0ba-a4cc6acd2d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:55.243789 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a84b6cda-9298-42d7-b0ba-a4cc6acd2d01 None None] MFA Rules not processed for user `c2b2bc070ec1473d87e36e4bf497a46f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:55.245006 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a84b6cda-9298-42d7-b0ba-a4cc6acd2d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:55.307164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1445/2881] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:55.319019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ea77d-f090-4611-a4e9-361f9ff2e2cb admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:55.320803 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-454ea77d-f090-4611-a4e9-361f9ff2e2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:55.377006 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ea77d-f090-4611-a4e9-361f9ff2e2cb admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:55.381142 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347e127-7e94-4251-b7b0-38b7bf5fca91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:55.381748 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347e127-7e94-4251-b7b0-38b7bf5fca91 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:55.382027 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347e127-7e94-4251-b7b0-38b7bf5fca91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:55.382302 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347e127-7e94-4251-b7b0-38b7bf5fca91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:55.390411 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347e127-7e94-4251-b7b0-38b7bf5fca91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:55.391858 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347e127-7e94-4251-b7b0-38b7bf5fca91 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:55.400624 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1437/2882] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:55.420145 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b6cda-9298-42d7-b0ba-a4cc6acd2d01 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:55.481285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b6cda-9298-42d7-b0ba-a4cc6acd2d01 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:55.485731 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:55.486357 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:55.486639 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:55.486917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] PATH_INFO: `/v3/users/c2b2bc070ec1473d87e36e4bf497a46f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:55.488709 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2b2bc070ec1473d87e36e4bf497a46f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:55.490602 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:55.529434 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:55.539603 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2b2bc070ec1473d87e36e4bf497a46f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:55.546917 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2b2bc070ec1473d87e36e4bf497a46f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:55.558895 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2b2bc070ec1473d87e36e4bf497a46f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:55.559578 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2b2bc070ec1473d87e36e4bf497a46f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:55.560205 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2b2bc070ec1473d87e36e4bf497a46f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:55.572247 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1446/2883] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:49:55 2022] DELETE /identity/v3/users/c2b2bc070ec1473d87e36e4bf497a46f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 07:49:55.576692 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff57722-d47c-49c0-82e5-3279051aaf57 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:55.576987 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff57722-d47c-49c0-82e5-3279051aaf57 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:55.577263 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff57722-d47c-49c0-82e5-3279051aaf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:55.596274 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-6ff57722-d47c-49c0-82e5-3279051aaf57 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:55.598226 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6ff57722-d47c-49c0-82e5-3279051aaf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:55.655030 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1438/2884] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:49:55.665422 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:55.666819 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:55.723021 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f475ecd-df41-430b-bea9-b8fe0f063450 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:55.726690 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b9b8d-1806-42c4-9622-f857edd3b26f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:55.727336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b9b8d-1806-42c4-9622-f857edd3b26f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:55.727621 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b9b8d-1806-42c4-9622-f857edd3b26f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:55.727899 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b9b8d-1806-42c4-9622-f857edd3b26f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:55.736843 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b9b8d-1806-42c4-9622-f857edd3b26f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:55.738240 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b9b8d-1806-42c4-9622-f857edd3b26f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:55.747545 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1447/2885] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:56.009217 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff57722-d47c-49c0-82e5-3279051aaf57 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:56.011019 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-6ff57722-d47c-49c0-82e5-3279051aaf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:56.023854 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff57722-d47c-49c0-82e5-3279051aaf57 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:56.028658 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61513058-d986-4e01-a2d5-4b665928e3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:56.029433 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61513058-d986-4e01-a2d5-4b665928e3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.029796 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61513058-d986-4e01-a2d5-4b665928e3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.030078 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61513058-d986-4e01-a2d5-4b665928e3e8 admin admin] PATH_INFO: `/v3/projects/068e70a4c3484902aba1ccefe590af73` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.031276 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61513058-d986-4e01-a2d5-4b665928e3e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=068e70a4c3484902aba1ccefe590af73)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:56.033196 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61513058-d986-4e01-a2d5-4b665928e3e8 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:56.072035 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:56.083391 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61513058-d986-4e01-a2d5-4b665928e3e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '068e70a4c3484902aba1ccefe590af73', 'request_id': 'req-61513058-d986-4e01-a2d5-4b665928e3e8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:56.088940 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61513058-d986-4e01-a2d5-4b665928e3e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '068e70a4c3484902aba1ccefe590af73', 'request_id': 'req-61513058-d986-4e01-a2d5-4b665928e3e8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:56.094978 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61513058-d986-4e01-a2d5-4b665928e3e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '068e70a4c3484902aba1ccefe590af73', 'request_id': 'req-61513058-d986-4e01-a2d5-4b665928e3e8'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:56.107381 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61513058-d986-4e01-a2d5-4b665928e3e8 admin admin] The token cache is being invalidate because project 068e70a4c3484902aba1ccefe590af73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:49:56.107669 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61513058-d986-4e01-a2d5-4b665928e3e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:56.108288 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-61513058-d986-4e01-a2d5-4b665928e3e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:56.109824 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1439/2886] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:49:56 2022] DELETE /identity/v3/projects/068e70a4c3484902aba1ccefe590af73 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:56.114764 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0b35e-7ee6-4ee4-b42b-bba0aba02002 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.115277 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0b35e-7ee6-4ee4-b42b-bba0aba02002 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.115390 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0b35e-7ee6-4ee4-b42b-bba0aba02002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.134735 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a1e0b35e-7ee6-4ee4-b42b-bba0aba02002 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:56.135978 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a1e0b35e-7ee6-4ee4-b42b-bba0aba02002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:56.193411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1448/2887] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:56.201180 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019a174-c14b-45a7-bb70-4fa1e16a429d None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.201473 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019a174-c14b-45a7-bb70-4fa1e16a429d None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.201762 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019a174-c14b-45a7-bb70-4fa1e16a429d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.221960 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-c019a174-c14b-45a7-bb70-4fa1e16a429d None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:56.223195 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-c019a174-c14b-45a7-bb70-4fa1e16a429d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:56.230973 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1440/2888] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:56.235060 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0b35e-7ee6-4ee4-b42b-bba0aba02002 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:56.245597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0b35e-7ee6-4ee4-b42b-bba0aba02002 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:56.250203 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea099f82-47fa-47a4-8a64-c1350b7eba5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:56.250863 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea099f82-47fa-47a4-8a64-c1350b7eba5c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.251119 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea099f82-47fa-47a4-8a64-c1350b7eba5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.251410 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea099f82-47fa-47a4-8a64-c1350b7eba5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.252101 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea099f82-47fa-47a4-8a64-c1350b7eba5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 07:49:56.252408 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea099f82-47fa-47a4-8a64-c1350b7eba5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:56.254736 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea099f82-47fa-47a4-8a64-c1350b7eba5c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:56.266063 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1449/2889] 10.222.0.40 () {58 vars in 1153 bytes} [Mon Aug 1 07:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 07:49:56.270676 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019a174-c14b-45a7-bb70-4fa1e16a429d None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:56.279819 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019a174-c14b-45a7-bb70-4fa1e16a429d None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:56.284552 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576af03-3b79-4937-9f1b-73b021bda508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:56.285131 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576af03-3b79-4937-9f1b-73b021bda508 admin admin] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.285410 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576af03-3b79-4937-9f1b-73b021bda508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.285682 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576af03-3b79-4937-9f1b-73b021bda508 admin admin] PATH_INFO: `/v3/projects` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.286344 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576af03-3b79-4937-9f1b-73b021bda508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:56.288265 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576af03-3b79-4937-9f1b-73b021bda508 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:56.316138 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1441/2890] 10.222.0.40 () {60 vars in 1153 bytes} [Mon Aug 1 07:49:56 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:56.320280 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea099f82-47fa-47a4-8a64-c1350b7eba5c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:56.330177 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea099f82-47fa-47a4-8a64-c1350b7eba5c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:56.370318 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:56.387164 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336ae0c-872a-4560-8e44-5b00c6cc83e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:56.387785 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336ae0c-872a-4560-8e44-5b00c6cc83e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.388074 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336ae0c-872a-4560-8e44-5b00c6cc83e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.388355 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336ae0c-872a-4560-8e44-5b00c6cc83e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.389018 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336ae0c-872a-4560-8e44-5b00c6cc83e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:56.390931 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336ae0c-872a-4560-8e44-5b00c6cc83e1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:56.411176 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1450/2891] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Aug 1 07:49:56 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 07:49:56.415267 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576af03-3b79-4937-9f1b-73b021bda508 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:56.426148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576af03-3b79-4937-9f1b-73b021bda508 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:56.430660 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1837e04-b1b7-4449-8e36-75ee0f8c1f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:56.431293 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1837e04-b1b7-4449-8e36-75ee0f8c1f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.431577 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1837e04-b1b7-4449-8e36-75ee0f8c1f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.431853 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1837e04-b1b7-4449-8e36-75ee0f8c1f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.432538 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1837e04-b1b7-4449-8e36-75ee0f8c1f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:56.434122 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1837e04-b1b7-4449-8e36-75ee0f8c1f60 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:56.445119 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1442/2892] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 07:49:56.448977 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336ae0c-872a-4560-8e44-5b00c6cc83e1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:56.458837 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336ae0c-872a-4560-8e44-5b00c6cc83e1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:56.463476 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5cf17-c638-4407-8be3-6da74b245df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:56.464076 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5cf17-c638-4407-8be3-6da74b245df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.464365 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5cf17-c638-4407-8be3-6da74b245df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.464648 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5cf17-c638-4407-8be3-6da74b245df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.465342 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5cf17-c638-4407-8be3-6da74b245df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:56.466921 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5cf17-c638-4407-8be3-6da74b245df8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:56.477442 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1451/2893] 10.222.0.40 () {58 vars in 1122 bytes} [Mon Aug 1 07:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 07:49:56.481318 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1837e04-b1b7-4449-8e36-75ee0f8c1f60 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:56.491817 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1837e04-b1b7-4449-8e36-75ee0f8c1f60 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:56.496260 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c476fa4-8341-4e16-a1f3-feb6047e0999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:56.496901 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c476fa4-8341-4e16-a1f3-feb6047e0999 admin admin] REQUEST_METHOD: `PUT` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.497184 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c476fa4-8341-4e16-a1f3-feb6047e0999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.497457 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c476fa4-8341-4e16-a1f3-feb6047e0999 admin admin] PATH_INFO: `/v3/projects/acd06a3f76e849edac5424d9cf91afcf/users/99e0f717efd84e01af48fd4c172c1fe5/roles/d8e3b1d89cfa4e0fb43b360a31b56605` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.512654 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c476fa4-8341-4e16-a1f3-feb6047e0999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acd06a3f76e849edac5424d9cf91afcf, user_id=99e0f717efd84e01af48fd4c172c1fe5, role_id=d8e3b1d89cfa4e0fb43b360a31b56605)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:56.515252 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c476fa4-8341-4e16-a1f3-feb6047e0999 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:56.528487 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1443/2894] 10.222.0.40 () {60 vars in 1483 bytes} [Mon Aug 1 07:49:56 2022] PUT /identity/v3/projects/acd06a3f76e849edac5424d9cf91afcf/users/99e0f717efd84e01af48fd4c172c1fe5/roles/d8e3b1d89cfa4e0fb43b360a31b56605 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:56.539387 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd8730-99fa-4c05-8adb-5a0ec8aae744 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.539684 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd8730-99fa-4c05-8adb-5a0ec8aae744 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.539968 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd8730-99fa-4c05-8adb-5a0ec8aae744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.568238 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4dbd8730-99fa-4c05-8adb-5a0ec8aae744 None None] MFA Rules not processed for user `99e0f717efd84e01af48fd4c172c1fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:56.569462 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4dbd8730-99fa-4c05-8adb-5a0ec8aae744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:56.632448 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1452/2895] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 07:49:56.643959 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c476fa4-8341-4e16-a1f3-feb6047e0999 admin admin] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:56.645969 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-2c476fa4-8341-4e16-a1f3-feb6047e0999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:56.702769 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c476fa4-8341-4e16-a1f3-feb6047e0999 admin admin] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:56.706599 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44676924-ec4e-4bee-a575-15ab85c2cb82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '495c489f00d74b53ac02c2cbbb323c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:56.707240 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44676924-ec4e-4bee-a575-15ab85c2cb82 service cinder] REQUEST_METHOD: `GET` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.707537 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44676924-ec4e-4bee-a575-15ab85c2cb82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.707906 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44676924-ec4e-4bee-a575-15ab85c2cb82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.716176 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44676924-ec4e-4bee-a575-15ab85c2cb82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:56.717603 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44676924-ec4e-4bee-a575-15ab85c2cb82 service cinder] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:56.727162 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1444/2896] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:56.752467 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd8730-99fa-4c05-8adb-5a0ec8aae744 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:56.816138 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd8730-99fa-4c05-8adb-5a0ec8aae744 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:56.820678 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:56.821312 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.821631 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.821914 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] PATH_INFO: `/v3/users/99e0f717efd84e01af48fd4c172c1fe5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.823128 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e0f717efd84e01af48fd4c172c1fe5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:56.825244 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:56.870409 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e0f717efd84e01af48fd4c172c1fe5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:56.877661 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e0f717efd84e01af48fd4c172c1fe5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:56.879336 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:56.892618 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e0f717efd84e01af48fd4c172c1fe5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:56.893349 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e0f717efd84e01af48fd4c172c1fe5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:56.894285 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e0f717efd84e01af48fd4c172c1fe5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:56.906202 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1453/2897] 10.222.0.40 () {58 vars in 1224 bytes} [Mon Aug 1 07:49:56 2022] DELETE /identity/v3/users/99e0f717efd84e01af48fd4c172c1fe5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:49:56.911278 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea6085-f15b-48aa-8fbb-107b3e920591 None None] REQUEST_METHOD: `POST` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:56.911573 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea6085-f15b-48aa-8fbb-107b3e920591 None None] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:56.911859 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea6085-f15b-48aa-8fbb-107b3e920591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:56.932044 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.auth.core [None req-75ea6085-f15b-48aa-8fbb-107b3e920591 None None] MFA Rules not processed for user `aae3d76a87c24d97b28248f570423938`. Rule list: `[]` (Enabled: `True`). {{(pid=80159) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 07:49:56.933290 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-75ea6085-f15b-48aa-8fbb-107b3e920591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:56.994877 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1445/2898] 10.222.0.40 () {58 vars in 958 bytes} [Mon Aug 1 07:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 07:49:57.004599 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:57.006012 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:57.063278 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38044922-c39e-4971-a55a-6f7c6b1c32f5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:57.067103 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7868203-07ea-486d-8b21-c204bc08af58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '138ede2854894a3b995435980965911c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:57.067653 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7868203-07ea-486d-8b21-c204bc08af58 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:57.067941 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7868203-07ea-486d-8b21-c204bc08af58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:57.068206 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7868203-07ea-486d-8b21-c204bc08af58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:57.077209 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7868203-07ea-486d-8b21-c204bc08af58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:57.078676 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7868203-07ea-486d-8b21-c204bc08af58 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:57.089386 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1454/2899] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:49:57.355846 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea6085-f15b-48aa-8fbb-107b3e920591 None None] Authenticating user token {{(pid=80159) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:49:57.357316 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.fernet_utils [None req-75ea6085-f15b-48aa-8fbb-107b3e920591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80159) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:49:57.364642 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:49:57.370295 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea6085-f15b-48aa-8fbb-107b3e920591 None None] Validating token access rules against request {{(pid=80159) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:49:57.376071 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aae3d76a87c24d97b28248f570423938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9fa482966924d01a3dc7809fa925311', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80159) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:49:57.376800 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:49:57.377235 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:49:57.377627 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4 admin admin] PATH_INFO: `/v3/projects/acd06a3f76e849edac5424d9cf91afcf` {{(pid=80159) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:49:57.379945 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acd06a3f76e849edac5424d9cf91afcf)` {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:49:57.381863 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4 admin admin] RBAC: Authorization granted {{(pid=80159) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:49:57.428480 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acd06a3f76e849edac5424d9cf91afcf', 'request_id': 'req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:57.433891 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acd06a3f76e849edac5424d9cf91afcf', 'request_id': 'req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:57.441148 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acd06a3f76e849edac5424d9cf91afcf', 'request_id': 'req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4'} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:57.453141 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4 admin admin] The token cache is being invalidate because project acd06a3f76e849edac5424d9cf91afcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80159) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 07:49:57.453429 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:57.454047 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG keystone.notifications [None req-58e9ae91-b5cd-4e20-be43-21c3b99d9fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80159) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 07:49:57.455570 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: [pid: 80159|app: 0|req: 1446/2900] 10.222.0.40 () {58 vars in 1233 bytes} [Mon Aug 1 07:49:57 2022] DELETE /identity/v3/projects/acd06a3f76e849edac5424d9cf91afcf => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 07:50:07.085129 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=37,keystone:DELETE=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:50:07.450068 dv-cinder-733062-31-iscsi devstack@keystone.service[80159]: DEBUG dbcounter [-] [80159] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 07:50:20.941071 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7868203-07ea-486d-8b21-c204bc08af58 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 07:50:20.942588 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7868203-07ea-486d-8b21-c204bc08af58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:50:21.012341 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7868203-07ea-486d-8b21-c204bc08af58 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 07:50:21.071791 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d7615-de0a-492d-9308-b12a407733fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928c5d1284b74f1291c031323bb03a20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ddf7d6f2424a85a741d017497b49c9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 07:50:21.072411 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d7615-de0a-492d-9308-b12a407733fa service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 07:50:21.072699 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d7615-de0a-492d-9308-b12a407733fa service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 07:50:21.072981 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d7615-de0a-492d-9308-b12a407733fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 07:50:21.074733 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f7d7615-de0a-492d-9308-b12a407733fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 07:50:21.086439 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d7615-de0a-492d-9308-b12a407733fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 07:50:21.088045 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d7615-de0a-492d-9308-b12a407733fa service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 07:50:21.097829 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1455/2901] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Aug 1 07:50:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 07:50:31.093597 dv-cinder-733062-31-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=58 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}